Comment on Apple Silicon and macOS VMs: Faster LLM Inference with llama.cppComments−engzaanin1moThat makes sense. The title initially sounded like a general llama.cpp speedup on Apple Silicon, but if the improvement comes from fixing kernel selection inside Virtualization.framework VMs, that distinction is pretty important.−frabonacciOP1moagreed on the title. added more context below on the exact scope and why this is really a VM capability-reporting issue: https://news.ycombinator.com/item?id=49260087
Comments
That makes sense. The title initially sounded like a general llama.cpp speedup on Apple Silicon, but if the improvement comes from fixing kernel selection inside Virtualization.framework VMs, that distinction is pretty important.
agreed on the title. added more context below on the exact scope and why this is really a VM capability-reporting issue: https://news.ycombinator.com/item?id=49260087