Comment on MLC-LLM: GPT/Llama on consumer-class GPUs and phonesComments−TheObviousOne3yis there any integration with it to langchain?+Is there any optimization for LLM to run on RTX cards? 40XX,30XX I found out tha LLAMA.CPP is nice but I want to take advantage of my graphic cards also, and didn't found any documentations...−juliangoldsmith3yrllama has an OpenCL version, though I wasn't able to test it.
Comments
is there any integration with it to langchain?
+
Is there any optimization for LLM to run on RTX cards? 40XX,30XX I found out tha LLAMA.CPP is nice but I want to take advantage of my graphic cards also, and didn't found any documentations...
rllama has an OpenCL version, though I wasn't able to test it.