I don't understand peoples expectation. If our language skills were as non-ambiguous as a coding language, we'd have solved world hunger by now.
So why would we expect all these bizarre bizantine language models to all conform to how a request is both made, expected and massaged.
For awhile, I was getting bizarre opencode tool errors where the only problem was the model was passing in a "1.0" or "0.0" where the harness dutifully wanted an integer. Of course 0.0 is the same as an integer in practical operations.
I have one core complex task where there are a number of simple errors like this. The easiest thing for me was to just have a post-processing script that performs: lint > mark known fail-early results > fix common errors (all formatting or type errors right now) > relint > final pass / fail > pass to next step (retry with tweaks / user in the loop since I don't have the token budget to just blindly keep trying the same thing).
Comments
I don't understand peoples expectation. If our language skills were as non-ambiguous as a coding language, we'd have solved world hunger by now.
So why would we expect all these bizarre bizantine language models to all conform to how a request is both made, expected and massaged.
For awhile, I was getting bizarre opencode tool errors where the only problem was the model was passing in a "1.0" or "0.0" where the harness dutifully wanted an integer. Of course 0.0 is the same as an integer in practical operations.
I have one core complex task where there are a number of simple errors like this. The easiest thing for me was to just have a post-processing script that performs: lint > mark known fail-early results > fix common errors (all formatting or type errors right now) > relint > final pass / fail > pass to next step (retry with tweaks / user in the loop since I don't have the token budget to just blindly keep trying the same thing).