Grok 4 has about 99% accuracy in picking the right tools and making tool calls with proper arguments almost every single time.
Where did this number come from? What is "the right tool"? I find this extremely subjective. As most engineers know, there is no right tool, but mostly a compromise where you pick the least worst tool and choose what risks you're willing to manage or not.
I believe in this context it means "tool" as in the MCP definition, e.g. "of the catalog of MCP integrations, it doesn't try to use the playright one to browse the web, it'd use the AWS docs one directly"
This is just my speculation, though, as I've never used Grok anything
That's langchain terminology. LLMs usually are exposed to a set of tools. It's usually pretty obvious which are obvious, since there's only one tool that's even remotely associated with the task at hand.
Comments
Grok 4 has about 99% accuracy in picking the right tools and making tool calls with proper arguments almost every single time.
Where did this number come from? What is "the right tool"? I find this extremely subjective. As most engineers know, there is no right tool, but mostly a compromise where you pick the least worst tool and choose what risks you're willing to manage or not.
I believe in this context it means "tool" as in the MCP definition, e.g. "of the catalog of MCP integrations, it doesn't try to use the playright one to browse the web, it'd use the AWS docs one directly"
This is just my speculation, though, as I've never used Grok anything
Yeah, based on a previous comment, that makes sense. I am a little reassured that is what the author meant.
That's langchain terminology. LLMs usually are exposed to a set of tools. It's usually pretty obvious which are obvious, since there's only one tool that's even remotely associated with the task at hand.
Thanks for the info. This makes the article slightly less intolerable!