Stop using JSON for LLM structured outputnehmeailabs.com 2 points44za127 months ago1 commentSaveHideCopy link On HNComments−44za12OP7moFor simple extraction tasks, a delimiter-separated string uses 11 tokens vs 35 for JSON. Output tokens are the latency bottleneck.
Comments
For simple extraction tasks, a delimiter-separated string uses 11 tokens vs 35 for JSON. Output tokens are the latency bottleneck.