Most things require workarounds, some things aren't possible (or we haven't found workaround yet) and it's not as fast as CUDA. But stable-diffusion inference works, and so does textual inversion training. I was also able to run training of a T5 model with just a couple of tweaks.
I'd stick with PyTorch 1.12.1 for now. 1.13 has problems with backpropagation (I get NaN gradients now when I attempt CLIP-guided diffusion -- I think this applies to training too), and some einsum formulations are 50% slower (there is a patch to fix this; I expect it'll be merged soon), making big self-attention matmuls slow and consequently making stable-diffusion inference ~6% slower.
It doesn’t support a few things, and has bugs in a few of the functions but besides that it works OK. Need to make sure your batch size is high enough to fully utilise the GPU cores. I think it was maybe 2-4 times faster than the CPU in my case.
I've used it for Stable Diffusion on my M1 mbp, works with no problem. I am not a AI/ML person so my experience with M1 PyTorch is limited to just using SD.
Yeah works great, Tensorflow works great as well, but if using pip need to install tensorflow-metal and tensorflow-macos. Plus I always need to downgrade protobuf to 3.20. And couldn't get the C bindings for Tensorflow to work through metal, only through CPU.
Comments
is anyone able to comment on the experience of PyTorch's M1 support?
https://pytorch.org/docs/stable/notes/mps.html
In my very limited experience it's not a ton faster than using a CPU backend though.
Most things require workarounds, some things aren't possible (or we haven't found workaround yet) and it's not as fast as CUDA. But stable-diffusion inference works, and so does textual inversion training. I was also able to run training of a T5 model with just a couple of tweaks.
I'd stick with PyTorch 1.12.1 for now. 1.13 has problems with backpropagation (I get NaN gradients now when I attempt CLIP-guided diffusion -- I think this applies to training too), and some einsum formulations are 50% slower (there is a patch to fix this; I expect it'll be merged soon), making big self-attention matmuls slow and consequently making stable-diffusion inference ~6% slower.
It doesn’t support a few things, and has bugs in a few of the functions but besides that it works OK. Need to make sure your batch size is high enough to fully utilise the GPU cores. I think it was maybe 2-4 times faster than the CPU in my case.
If you use miniforge (https://github.com/conda-forge/miniforge) you can just 'pip install torch' and it works.
I think you mean “conda install torch”, no? Miniforge would only impact the conda installer?
Surprisingly, no. I just tried it:
- 'pip install torch' doesn't work inside of a plain Python env (not miniforge)
- 'conda install torch' doesn't work inside miniforge
- 'pip install torch' works inside miniforge
Huh! That’s curious! I wonder what causes the different behaviour!
What's different about miniforge in this regard? Does miniforge come with additional package indexes pre-configured?
Pretty much. Conda installed with miniforge uses a different channel, with a focus on having packages available for more architectures.
I've used it for Stable Diffusion on my M1 mbp, works with no problem. I am not a AI/ML person so my experience with M1 PyTorch is limited to just using SD.
Yeah works great, Tensorflow works great as well, but if using pip need to install tensorflow-metal and tensorflow-macos. Plus I always need to downgrade protobuf to 3.20. And couldn't get the C bindings for Tensorflow to work through metal, only through CPU.
Pretty good machine learning experience on M1.