Comment on State-of-the-Art Chatbot, Vicuna-7B, now runs on MacBook with GPU accelerationparentComments−nickthegreek3yI can run the 30b 4bit model on my m2 air that has 24gb of ram.−jpp4e3yHi nickthegreek!Could you tell me how you did that? Did you use FastChat or something else? Which model to download? What command to run?Thank you!!!−nickthegreek3yhttps://huggingface.co/Pi3141/alpaca-lora-30B-ggmlI believe I’m using alpaca.cpp with a command:./chat -m <bin filename>
Comments
I can run the 30b 4bit model on my m2 air that has 24gb of ram.
Hi nickthegreek!
Could you tell me how you did that? Did you use FastChat or something else? Which model to download? What command to run?
Thank you!!!
https://huggingface.co/Pi3141/alpaca-lora-30B-ggml
I believe I’m using alpaca.cpp with a command:
./chat -m <bin filename>