Comment on MLC-LLM: GPT/Llama on consumer-class GPUs and phonesparentComments−junrushao1994OP3yTVM Unity, the compiler used by MLC-LLM, does support CPU and SIMD instructions on each CPU backend via LLVM, but we haven't tried it out yet. I believe llama.cpp is the best option out of box at the moment.
Comments
TVM Unity, the compiler used by MLC-LLM, does support CPU and SIMD instructions on each CPU backend via LLVM, but we haven't tried it out yet. I believe llama.cpp is the best option out of box at the moment.