Probably not. But if it’s the new data providing the advantage then you’re not exactly better off having the old data and the model vs. just having the model.
The idea would be that another group could fork it and continue adding to the dataset on their own.
As opposed to not being able to fork it at all because an "open source" model actually just means "you are allowed to use this particular release of our mystery box."
You do not need the original dataset to train the model on an additional dataset
Maybe I misunderstood your original question. To be clear, the process of modifying a trained model does not require the presence of the original data. You said “deleted” which perhaps I misinterpreted. You’re not “instantiating a new model from scratch” when you modify it. You’re continuing to train it where it left off.
What if you want to start with a subset of the original data? Like you've trained a model, and then later said "You know, this new data we're adding is great, but maybe pulling all those comments from 4chan earlier was a mistake," wouldn't that require starting fresh with access to the actual data?
Comments
Probably not. But if it’s the new data providing the advantage then you’re not exactly better off having the old data and the model vs. just having the model.
The idea would be that another group could fork it and continue adding to the dataset on their own.
As opposed to not being able to fork it at all because an "open source" model actually just means "you are allowed to use this particular release of our mystery box."
You do not need the original dataset to train the model on an additional dataset
Maybe I misunderstood your original question. To be clear, the process of modifying a trained model does not require the presence of the original data. You said “deleted” which perhaps I misinterpreted. You’re not “instantiating a new model from scratch” when you modify it. You’re continuing to train it where it left off.
What if you want to start with a subset of the original data? Like you've trained a model, and then later said "You know, this new data we're adding is great, but maybe pulling all those comments from 4chan earlier was a mistake," wouldn't that require starting fresh with access to the actual data?
Technically correct but not a very realistic request / approach.
The general idea is to get as good of a mastery of language as possible, generally, and then fine tune to specialize on tasks