Comment on Mesh LLM: distributed AI computing on irohparentComments−embedding-shape2mobecause they feature 800Gbps of memory bandwidth. About twice what's available on the top end of M5Ouch, about half of the memory bandwidth of a dedicated GPU though :/ Running LLMs on Apple hardware still doesn't make any sense to me.
Comments
Ouch, about half of the memory bandwidth of a dedicated GPU though :/ Running LLMs on Apple hardware still doesn't make any sense to me.