yeah agreed, its sad to me that we're 2 months after llama and "nobody" is seemingly doing any advances of fine tuning on models with more than 7B or 13B parameters.
I have 64gb RAM (not gpu just normal), I’d like to see proof of concepts that the bigger models can be fine tuned and have far more accepted results, or to know if we’re completely going the wrong direction with this
Probably because it’s extremely affordable to train the smaller models.
If I had the gumption (and a data set) I could afford to spend a few hundred bucks to fine tune a model for shits and giggles and I’m just a Random Internet Dude.
I’m all for it, Any Day Now™ I have this idea I want to try and having these people do all this optimization work will probably make it affordable to attempt given I don’t actually know what I’m doing so there will be a whole lot of “yeah, that doesn’t work” going on.
I find, since 30B models are actually quite usable locally if you have good hardware, that I really want something like a Vicuna 30B. That would be amazing. I can only run 65B locally at a speed of 1 token per second, which is too slow to be usable unfortunately.
Comments
yeah agreed, its sad to me that we're 2 months after llama and "nobody" is seemingly doing any advances of fine tuning on models with more than 7B or 13B parameters.
I have 64gb RAM (not gpu just normal), I’d like to see proof of concepts that the bigger models can be fine tuned and have far more accepted results, or to know if we’re completely going the wrong direction with this
Probably because it’s extremely affordable to train the smaller models.
If I had the gumption (and a data set) I could afford to spend a few hundred bucks to fine tune a model for shits and giggles and I’m just a Random Internet Dude.
I’m all for it, Any Day Now™ I have this idea I want to try and having these people do all this optimization work will probably make it affordable to attempt given I don’t actually know what I’m doing so there will be a whole lot of “yeah, that doesn’t work” going on.
I find, since 30B models are actually quite usable locally if you have good hardware, that I really want something like a Vicuna 30B. That would be amazing. I can only run 65B locally at a speed of 1 token per second, which is too slow to be usable unfortunately.