Like a torrent network but for training. That would be cool. The only question is how do you merge changes made by nodes (clients) across the network?
Clients could be incentivised to train as they are with crypto, but instead of mining, it's model training and in return they get "coin". Like making crypto mining useful.
While i understand the motives, they should just train models bits for free, a-la SETI@home. Then let people pick and choose which collections to use to their chatbot. The crypto aspect only adds complexity and wrong incentives that would hold it back. After all it's not possible to verify that each node truly trained the network and is not returning garbage (multiple trainings can be involved, which would make the whole thing slower).
Saw that - a real cool project! Thanks for sharing! Now the question is how to motivate people to train models?
I am thinking of somehow replacing crypto currency hashing with model training and inference. Somehow a training topic and data get fed into the network - naturally it somehow must be filtered to avoid weird crap - and queries are somehow run. Imagine all the crypto mining farms and gpus switching over to training massive models.
Yes, in a sense. It won't work well for distribution of training on a single subject, but you distribute training based on subject and then combine LoRAs.
Perhaps this could work like a set of hashed and trusted data sets split but subject and topic? Each node downloads one at random and trains against that single subset of a topic or something.
You can, like, just do weighted average. It works.
Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Models (this is Tim Dettmers's previous work!)
These ELMs can be added and removed to update data coverage, ensembled to generalize to new domains, or averaged to collapse back to a single LM for efficient inference.
So that means a massively distributed model training network with cryptocurrency like incentives is incoming? Where and how to begin? This could free up companies such as openai and potentially lead to the first agi.
(mentioning crypto because that will motivate switch hordes of miners that already have the gpu power available)
BTM is very promising, but it is unclear how much it scales, let alone "massively". The paper scaled it to 64 domains and it worked, but you probably want more than 64 nodes.
Since domain is actually important to its performance, you can't randomly split to 64 pieces, see Table 4 of the paper, "Domain expert ensemble outperforms random split ensemble". Performance difference is large.
So if you want to begin, I would start by researching how to scale domain split.
We already have exactly that for stable diffusion with Civitai.com. People have published a variety of LoRAs for different subjects just as you describe. The local LLM community is very much following the lead of the stable diffusion community in terms of how it's organizing, so I expect that we'll see a proliferation of domain LoRAs being published on an aggregator for LLM stuff before too long.
Some of the people releasing popular models/LoRAs on civitai do alright via Patreon in addition to getting a lot of praise from the SD community, that seems like incentive to me.
Comments
Since Loras are additive, is it possible to use them to do distributed retraining on a model, or even train an entire model bit by bit?
Like a torrent network but for training. That would be cool. The only question is how do you merge changes made by nodes (clients) across the network?
Clients could be incentivised to train as they are with crypto, but instead of mining, it's model training and in return they get "coin". Like making crypto mining useful.
While i understand the motives, they should just train models bits for free, a-la SETI@home. Then let people pick and choose which collections to use to their chatbot. The crypto aspect only adds complexity and wrong incentives that would hold it back. After all it's not possible to verify that each node truly trained the network and is not returning garbage (multiple trainings can be involved, which would make the whole thing slower).
https://github.com/bigscience-workshop/petals
Saw that - a real cool project! Thanks for sharing! Now the question is how to motivate people to train models?
I am thinking of somehow replacing crypto currency hashing with model training and inference. Somehow a training topic and data get fed into the network - naturally it somehow must be filtered to avoid weird crap - and queries are somehow run. Imagine all the crypto mining farms and gpus switching over to training massive models.
Yes, in a sense. It won't work well for distribution of training on a single subject, but you distribute training based on subject and then combine LoRAs.
How do you combine them?
Perhaps this could work like a set of hashed and trusted data sets split but subject and topic? Each node downloads one at random and trains against that single subset of a topic or something.
You can, like, just do weighted average. It works.
Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Models (this is Tim Dettmers's previous work!)
https://arxiv.org/abs/2208.03306
So that means a massively distributed model training network with cryptocurrency like incentives is incoming? Where and how to begin? This could free up companies such as openai and potentially lead to the first agi.
(mentioning crypto because that will motivate switch hordes of miners that already have the gpu power available)
BTM is very promising, but it is unclear how much it scales, let alone "massively". The paper scaled it to 64 domains and it worked, but you probably want more than 64 nodes.
Since domain is actually important to its performance, you can't randomly split to 64 pieces, see Table 4 of the paper, "Domain expert ensemble outperforms random split ensemble". Performance difference is large.
So if you want to begin, I would start by researching how to scale domain split.
We already have exactly that for stable diffusion with Civitai.com. People have published a variety of LoRAs for different subjects just as you describe. The local LLM community is very much following the lead of the stable diffusion community in terms of how it's organizing, so I expect that we'll see a proliferation of domain LoRAs being published on an aggregator for LLM stuff before too long.
I don't think the two concepts are similar. I see no incentive for people to train for civitai and find no particular use for the generated content.
Edit: actually, some of that content looks suspicious.
Some of the people releasing popular models/LoRAs on civitai do alright via Patreon in addition to getting a lot of praise from the SD community, that seems like incentive to me.
Porn is a major reason that people train loras. The beauty of them is that people can pick and choose multiple loras to build truly custom ai
I think there may be stuff worse than porn lurking around there.
I just had the same discussion with a friend, I'm pretty curious about this