I can't be certain, I'm not at all an AI engineer or math guy, but I think at the "wake up" point you equalize instances.
Like during 'sleep' some list of functions/operations `m` are applied to model weights `n` producing a new model, `n + 1`.
Wouldn't you just clone `n + 1`, send it to work, and start a new training run `m + 1` to make `n + 2`?
This was my first idea as well. Keep training continuously and redeploy clones after each cycle. From a layman perspective this seems reasonable :thinking:
You can't realistically keep training the same model forever, or it will start forgetting things it knew before. The proper name for this is "catastrophic forgetting".
Comments
I can't be certain, I'm not at all an AI engineer or math guy, but I think at the "wake up" point you equalize instances. Like during 'sleep' some list of functions/operations `m` are applied to model weights `n` producing a new model, `n + 1`. Wouldn't you just clone `n + 1`, send it to work, and start a new training run `m + 1` to make `n + 2`?
This was my first idea as well. Keep training continuously and redeploy clones after each cycle. From a layman perspective this seems reasonable :thinking:
You can't realistically keep training the same model forever, or it will start forgetting things it knew before. The proper name for this is "catastrophic forgetting".