I did not imply harnesses were useless, just that they're merely a functional part of running an LLM. But it's ticking a checkbox. If the harness can support LLM capabilities (tool calling, reading the screen, visual interaction), it's reached the limit of its potential. Like a JSON parser - can it do the job or not?
Astra didn't solve Arc-AGI-3 because of its phenomenal harness, just a harness was needed to support its capabilities.
Giving Astra's harness to a different LLM would not have improved said LLMs ability to solve the task in a meaningful way.
I feel like this is important to point out, seeing how I'm commenting under an article about how a harness is valued at $50B.
Imagine valuing a JSON parser at $50B on the basis that most communication can't be read without one.
Comments
I did not imply harnesses were useless, just that they're merely a functional part of running an LLM. But it's ticking a checkbox. If the harness can support LLM capabilities (tool calling, reading the screen, visual interaction), it's reached the limit of its potential. Like a JSON parser - can it do the job or not?
Astra didn't solve Arc-AGI-3 because of its phenomenal harness, just a harness was needed to support its capabilities.
Giving Astra's harness to a different LLM would not have improved said LLMs ability to solve the task in a meaningful way.
I feel like this is important to point out, seeing how I'm commenting under an article about how a harness is valued at $50B.
Imagine valuing a JSON parser at $50B on the basis that most communication can't be read without one.