The issue is not in function calls but HOW MCP got designed here and you are using.
Most MCP are replicating API. Returning blobs of data.
1. This is using a lot of input context in formating as JSON and escaping a Json inside already a JSON.
2. This contain a lot of irrelevant information that you can same on it.
So the issue is the MCP tool. It should instead flaten the data as possible as it's going back again thru JSON Encoding. And if needed remove some fields.
So MCP SAAS here are mainly API gateways.
That brings this noise!
And most of ALL they are not optimizing MCP's.
This is what GraphQL was designed for. Only select fields you really need. We've built an OSS Gateway that turns a collection of GraphQL queries into an MCP server to make this simple: https://wundergraph.com/mcp-gateway
MCP doesn't help but filtering is not always a good solution - sometimes you just need the agent to process a lot of data.
In that scenario running code on the data with minimum evaluation of the data (eg. a schema with explanation) is a much better approach and it will scale up to use cases of a certain complexity.
Even this system is not perfect: once your data definition and orchestration grow to big you'll face the same problems.
This should allow you to scale to pretty complex problems though, while the naive approach of just embedding API responses in the chat fails soon (I run into this issue frequently, maintaining a relatively simple systems with a few tool calls).
The only proper solution is reproducing the level of granularity of human decisions in code and call this "decisional system" from an LLM (which would be then reduced to a mere language interface between human language and the internal system).
Easier said than done, though.
Just for fun, I used ChatGPT to reverse a string as my first test of using their API. I was amused at how much work it took to get the LLM to give me just the reversed string, and even then I didn't feel I could fully trust it. I learned my lesson, and now I have multiple LLMs check to see of the string has actually been reversed. Soon I'll be spinning up a data center to host the GPUs necessary to correctly count the number of Rs in strawberry.
Comments
The issue is not in function calls but HOW MCP got designed here and you are using.
Most MCP are replicating API. Returning blobs of data.
1. This is using a lot of input context in formating as JSON and escaping a Json inside already a JSON. 2. This contain a lot of irrelevant information that you can same on it.
So the issue is the MCP tool. It should instead flaten the data as possible as it's going back again thru JSON Encoding. And if needed remove some fields.
So MCP SAAS here are mainly API gateways.
That brings this noise! And most of ALL they are not optimizing MCP's.
This is what GraphQL was designed for. Only select fields you really need. We've built an OSS Gateway that turns a collection of GraphQL queries into an MCP server to make this simple: https://wundergraph.com/mcp-gateway
MCP doesn't help but filtering is not always a good solution - sometimes you just need the agent to process a lot of data.
In that scenario running code on the data with minimum evaluation of the data (eg. a schema with explanation) is a much better approach and it will scale up to use cases of a certain complexity.
Even this system is not perfect: once your data definition and orchestration grow to big you'll face the same problems.
This should allow you to scale to pretty complex problems though, while the naive approach of just embedding API responses in the chat fails soon (I run into this issue frequently, maintaining a relatively simple systems with a few tool calls).
The only proper solution is reproducing the level of granularity of human decisions in code and call this "decisional system" from an LLM (which would be then reduced to a mere language interface between human language and the internal system). Easier said than done, though.
Isn't it a model problem that they don't respect complex json schemas?
Just for fun, I used ChatGPT to reverse a string as my first test of using their API. I was amused at how much work it took to get the LLM to give me just the reversed string, and even then I didn't feel I could fully trust it. I learned my lesson, and now I have multiple LLMs check to see of the string has actually been reversed. Soon I'll be spinning up a data center to host the GPUs necessary to correctly count the number of Rs in strawberry.