JSON retries generated 7x to 14x more output tokens than the args path.
But!
On PowerShell, the cost gap between args and JSON was 9x. On Bash, it collapsed to 1.5x.
The conclusion of the post is "cli args are better than JSON", but the argument for why only holds up on Windows. This post says more about Powershell than anything else.
Comments
Some big findings:
But!
The conclusion of the post is "cli args are better than JSON", but the argument for why only holds up on Windows. This post says more about Powershell than anything else.
Yes it's a shame they didn't show examples of how string quoting was causing retries in Powershell vs Bash.
The title of the article is misleading and is the wrong takeaway from the experiment.