Like a torrent network but for training. That would be cool. The only question is how do you merge changes made by nodes (clients) across the network?
Clients could be incentivised to train as they are with crypto, but instead of mining, it's model training and in return they get "coin". Like making crypto mining useful.
While i understand the motives, they should just train models bits for free, a-la SETI@home. Then let people pick and choose which collections to use to their chatbot. The crypto aspect only adds complexity and wrong incentives that would hold it back. After all it's not possible to verify that each node truly trained the network and is not returning garbage (multiple trainings can be involved, which would make the whole thing slower).
Saw that - a real cool project! Thanks for sharing! Now the question is how to motivate people to train models?
I am thinking of somehow replacing crypto currency hashing with model training and inference. Somehow a training topic and data get fed into the network - naturally it somehow must be filtered to avoid weird crap - and queries are somehow run. Imagine all the crypto mining farms and gpus switching over to training massive models.
Comments
Like a torrent network but for training. That would be cool. The only question is how do you merge changes made by nodes (clients) across the network?
Clients could be incentivised to train as they are with crypto, but instead of mining, it's model training and in return they get "coin". Like making crypto mining useful.
While i understand the motives, they should just train models bits for free, a-la SETI@home. Then let people pick and choose which collections to use to their chatbot. The crypto aspect only adds complexity and wrong incentives that would hold it back. After all it's not possible to verify that each node truly trained the network and is not returning garbage (multiple trainings can be involved, which would make the whole thing slower).
https://github.com/bigscience-workshop/petals
Saw that - a real cool project! Thanks for sharing! Now the question is how to motivate people to train models?
I am thinking of somehow replacing crypto currency hashing with model training and inference. Somehow a training topic and data get fed into the network - naturally it somehow must be filtered to avoid weird crap - and queries are somehow run. Imagine all the crypto mining farms and gpus switching over to training massive models.