Comment on Local LLM inference – impressive but too hard to work withparentComments−ijk1yThere's local applications of parallel processing; your average chatbot wouldn't use it, but a research bot with multiple simultaneous queries will, for example.Better local beamsearch would be really nice to have, though.
Comments
There's local applications of parallel processing; your average chatbot wouldn't use it, but a research bot with multiple simultaneous queries will, for example.
Better local beamsearch would be really nice to have, though.