So that means a massively distributed model training network with cryptocurrency like incentives is incoming? Where and how to begin? This could free up companies such as openai and potentially lead to the first agi.
(mentioning crypto because that will motivate switch hordes of miners that already have the gpu power available)
BTM is very promising, but it is unclear how much it scales, let alone "massively". The paper scaled it to 64 domains and it worked, but you probably want more than 64 nodes.
Since domain is actually important to its performance, you can't randomly split to 64 pieces, see Table 4 of the paper, "Domain expert ensemble outperforms random split ensemble". Performance difference is large.
So if you want to begin, I would start by researching how to scale domain split.
We already have exactly that for stable diffusion with Civitai.com. People have published a variety of LoRAs for different subjects just as you describe. The local LLM community is very much following the lead of the stable diffusion community in terms of how it's organizing, so I expect that we'll see a proliferation of domain LoRAs being published on an aggregator for LLM stuff before too long.
Some of the people releasing popular models/LoRAs on civitai do alright via Patreon in addition to getting a lot of praise from the SD community, that seems like incentive to me.
Comments
So that means a massively distributed model training network with cryptocurrency like incentives is incoming? Where and how to begin? This could free up companies such as openai and potentially lead to the first agi.
(mentioning crypto because that will motivate switch hordes of miners that already have the gpu power available)
BTM is very promising, but it is unclear how much it scales, let alone "massively". The paper scaled it to 64 domains and it worked, but you probably want more than 64 nodes.
Since domain is actually important to its performance, you can't randomly split to 64 pieces, see Table 4 of the paper, "Domain expert ensemble outperforms random split ensemble". Performance difference is large.
So if you want to begin, I would start by researching how to scale domain split.
We already have exactly that for stable diffusion with Civitai.com. People have published a variety of LoRAs for different subjects just as you describe. The local LLM community is very much following the lead of the stable diffusion community in terms of how it's organizing, so I expect that we'll see a proliferation of domain LoRAs being published on an aggregator for LLM stuff before too long.
I don't think the two concepts are similar. I see no incentive for people to train for civitai and find no particular use for the generated content.
Edit: actually, some of that content looks suspicious.
Some of the people releasing popular models/LoRAs on civitai do alright via Patreon in addition to getting a lot of praise from the SD community, that seems like incentive to me.
Porn is a major reason that people train loras. The beauty of them is that people can pick and choose multiple loras to build truly custom ai
I think there may be stuff worse than porn lurking around there.