Do you think this is inevitable? It sounds like, if distributed LLMs are technically feasible to achieve, it will eventually happen. Maybe that's an unknown whether it can be solved at all, but I imagine there are enough people working on the problem that they will find a break-through one way or the other. LLMs themselves could participate in solving it.

Edit: Oh I just saw the Git repo:

exo: Run your own AI cluster at home with everyday devices.

So the "distributed problem" is in the process of being solved. Impressive.