Seriously, I can't stand these tests of 1000 input tokens generating 1 million output tokens. You can't really convey anything interesting in 100 words, so like almost who cares what it produces? No one is providing value if the extent of their contribution is so small.
Much more interesting is providing a million tokens of meaningful input and getting 1000 tokens out (high level critique of a detailed design doc, finding a subtle bug in a big codebase, etc).
I think both can be interesting, but surely the 100 words will lead to a lot of variability in the exact shape of the result because so much of the spec will be left up to the LLM. However, it is still interesting to judge the quality of the result and how self-consistent it is. Not by looking at the actual webpage, but by looking at the code and whether it is nicely organized, makes proper use of abstractions, and in general doesn't feel like spaghetti.
Great - can you show us some examples of these? Every time this topic comes up here it’s full of people saying “you’re using it wrong” and apparently it should just be intuitive. Yet all the material is like this.
Comments
Seriously, I can't stand these tests of 1000 input tokens generating 1 million output tokens. You can't really convey anything interesting in 100 words, so like almost who cares what it produces? No one is providing value if the extent of their contribution is so small.
Much more interesting is providing a million tokens of meaningful input and getting 1000 tokens out (high level critique of a detailed design doc, finding a subtle bug in a big codebase, etc).
I think both can be interesting, but surely the 100 words will lead to a lot of variability in the exact shape of the result because so much of the spec will be left up to the LLM. However, it is still interesting to judge the quality of the result and how self-consistent it is. Not by looking at the actual webpage, but by looking at the code and whether it is nicely organized, makes proper use of abstractions, and in general doesn't feel like spaghetti.
Great - can you show us some examples of these? Every time this topic comes up here it’s full of people saying “you’re using it wrong” and apparently it should just be intuitive. Yet all the material is like this.