Not a Mac Studio but I use a basic Macbook Pro laptop with 24 GB of RAM (16 usable as VRAM) and I can run a number of models on it at decent speed, my main bottleneck is context window size, but if I am asking single purpose questions I am fine.
Yeah. I'm currently on an Mac Mini m2 Pro with 32GB or ram, and I was so curious how much more I could get out of the Apple ecosystem. Thanks for your perspective.
The most I ran was a GPT 20b model, I also run SDXL it runs rather quickly via the Draw Things app. There's an 8 step LoRa that lets you generate images in just 8 steps.
Comments
Not a Mac Studio but I use a basic Macbook Pro laptop with 24 GB of RAM (16 usable as VRAM) and I can run a number of models on it at decent speed, my main bottleneck is context window size, but if I am asking single purpose questions I am fine.
Yeah. I'm currently on an Mac Mini m2 Pro with 32GB or ram, and I was so curious how much more I could get out of the Apple ecosystem. Thanks for your perspective.
What models are you running?
The most I ran was a GPT 20b model, I also run SDXL it runs rather quickly via the Draw Things app. There's an 8 step LoRa that lets you generate images in just 8 steps.