Ask HN: Case Study on Training LLMs on Apple M2 Ultra Neural Engine?

1 pointpritambarhate2 years ago1 comment

Has someone here come across a case study on training LLMs on Apple M2 Ultra Neural Engine? I wanted to know how it would compare to training LLMs on GPUs like H100.

Considering the cost and shortage of H100, can Mac Pro Ultras be used for training LLMs? I mean people are trying to do it on SuperComputers using CPUs (https://news.ycombinator.com/item?id=40348371) surely someone must have tried using Apple Silicon Neural Engine.

I tried searching for it, but didn't find anything proper.

talldayo 2y

The Neural Engine itself on those is pretty much non-comparable to something like an H100. The GPU is much more suitable for training, or even the CPU probably.

Comments