32GB M1 Max is taking 25 seconds on the exact same prompt as in the example.
Edit: it seems the "per second" requires the `--continuous` flag to bypass the initial startup time. With that, I'm now seeing the ~1 second per image time (if initial startup time is ignored).
I’m probably missing something but if the bottleneck is disk read speed, wouldn’t it only take about 5-6 seconds to fill the entire 32GB memory from disk? I just googled and found a benchmark quoting 5,507 MB/s read on an M1 Max.
Comments
32GB M1 Max is taking 25 seconds on the exact same prompt as in the example.
Edit: it seems the "per second" requires the `--continuous` flag to bypass the initial startup time. With that, I'm now seeing the ~1 second per image time (if initial startup time is ignored).
What does bypass startup time really do? Does it keep everything in memory or something?
Probably, you have to load the weights from disk at some point.
That's exactly it. These models are huge.
I’m probably missing something but if the bottleneck is disk read speed, wouldn’t it only take about 5-6 seconds to fill the entire 32GB memory from disk? I just googled and found a benchmark quoting 5,507 MB/s read on an M1 Max.
PyTorch checkpoint is slow to load.
The diffusers format this repo uses should be faster, but there is still some overhead, yeah.
Yeah, the PyTorch disk format is pretty bad.