I've had a fair amount of success at work recently with treating LLMs - specifically OpenAI's GPT-4 with function calling - as modules in a larger system, helped along powerfully by the ability to output structured data.
Most systems need to be much faster than LLMs are today, and on current trends of efficiency and hardware improvements, will be for the next several years.
I think here I disagree with the author here though, and am happy to be a technological optimist - if LLMs are used modularly, what's to stop us in a few years (presumably still hardware requirement costs, on reflection) eventually having small, fast specialised LLMs for the things that we find them truly useful/irreplaceable?
Nothing's to stop us, and in fact we can do that now! This is basically what the post advocates for: replacing the LLM calls for task-specific things with smaller models. They just don't need to be LLMs.
Comments
I've had a fair amount of success at work recently with treating LLMs - specifically OpenAI's GPT-4 with function calling - as modules in a larger system, helped along powerfully by the ability to output structured data.
I think here I disagree with the author here though, and am happy to be a technological optimist - if LLMs are used modularly, what's to stop us in a few years (presumably still hardware requirement costs, on reflection) eventually having small, fast specialised LLMs for the things that we find them truly useful/irreplaceable?
Nothing's to stop us, and in fact we can do that now! This is basically what the post advocates for: replacing the LLM calls for task-specific things with smaller models. They just don't need to be LLMs.