Comment on State-of-the-Art Chatbot, Vicuna-7B, now runs on MacBook with GPU accelerationparentComments−wmf3yApple's unified memory should allow running large models like 65B that will not fit on a consumer GPU, but mostly I see people talking about the smaller 7B sizes that can run anywhere.
Comments
Apple's unified memory should allow running large models like 65B that will not fit on a consumer GPU, but mostly I see people talking about the smaller 7B sizes that can run anywhere.