Also, sometimes older models work fine, Opus 4.8/4.6/4.5 are worth trying.
I would go as far as saying they are sometimes better than the newer ones! It is impressive that Opus 5 can one-shot a complex website, but it is very annoying if I only asked it to write tests.
Last week, I asked Sonnet 5 to add one line to the documentation, commit, and push. Ten minutes later, I discovered it embarked on a side quest of exploring my whole Git history, different branches, etc.
It's fun to offer this advice but I think actually practicing it is nearly impossible in many settings.
If you are a solo developer who has the patience to tinker endlessly, this advice is probably fine.
If you are responsible for provisioning AI services in a team setting, this advice starts to fall apart rapidly. OAI and Anthropic might as well be oil and water when it comes to what tools and descriptions are most ideal.
Swapping the inference provider like it's some interchangeable module is a total fantasy in most real world settings.
Also, sometimes older models work fine
sometimes
My users are hoping for slightly more definitive results. "Usually" or even "often" would be much preferred.
You don't neccesarily have to fallback across models. Both Anthropic and AWS Bedrock provide the same models with the same API at different endpoints and different infra. So if you have a proxy endpoint to point users at, that specific fallback case is pretty easy.
I'm not saying that works in every business or billing scenario, though.
Failover saves the call, not the run. If the provider fails mid-agent run, the second provider starts without knowing which tool calls have already been written, and the retry on a non-idempotent write simply creates the record twice. Without an idempotency key for each tool call and a checkpoint after each write, two providers make you available, but not correctly.
Comments
Great reminder to be using multiple providers (i.e. Anthropic 1P, Bedrock, Vertex AI, Azure) with auto-fallback.
Bedrock has been fine throughout today; just pick two providers and you have much more stable Claude :)
Also, sometimes older models work fine, Opus 4.8/4.6/4.5 are worth trying.
I would go as far as saying they are sometimes better than the newer ones! It is impressive that Opus 5 can one-shot a complex website, but it is very annoying if I only asked it to write tests.
Last week, I asked Sonnet 5 to add one line to the documentation, commit, and push. Ten minutes later, I discovered it embarked on a side quest of exploring my whole Git history, different branches, etc.
It's fun to offer this advice but I think actually practicing it is nearly impossible in many settings.
If you are a solo developer who has the patience to tinker endlessly, this advice is probably fine.
If you are responsible for provisioning AI services in a team setting, this advice starts to fall apart rapidly. OAI and Anthropic might as well be oil and water when it comes to what tools and descriptions are most ideal.
Swapping the inference provider like it's some interchangeable module is a total fantasy in most real world settings.
My users are hoping for slightly more definitive results. "Usually" or even "often" would be much preferred.
You don't neccesarily have to fallback across models. Both Anthropic and AWS Bedrock provide the same models with the same API at different endpoints and different infra. So if you have a proxy endpoint to point users at, that specific fallback case is pretty easy.
I'm not saying that works in every business or billing scenario, though.
Never put all AI taxes in one basket!
Failover saves the call, not the run. If the provider fails mid-agent run, the second provider starts without knowing which tool calls have already been written, and the retry on a non-idempotent write simply creates the record twice. Without an idempotency key for each tool call and a checkpoint after each write, two providers make you available, but not correctly.