LoRA is an alternative to traditional fine tuning (which is usually done on specific layers as you mentioned).
To quote the LoRA paper[1]:
We hypothesize that the change in weights during model adaptation also has a low “intrinsic rank”, leading to our proposed Low-Rank Adaptation (LoRA) approach. LoRA allows us to train some dense layers in a neural network indirectly by optimizing rank decomposition matrices of the dense layers’ change during adaptation instead, while keeping the pre-trained weights frozen
It's truly revolutionary: It basically lets you create a very small "diff" which you apply yo an existing model and it is suddenly fine tuned. These diff models are very small (5M for example).
Or “multiplex” fine-tuning with inference, ie. do fine-tuning for 100ms, inference for 100ms, then tuning again… etc
Btw, is there a way to combine two or more models?
So for example, if I create 5 copies of a model, then fine-tune each copy with a different dataset -> can the 5 datasets be merged together somehow to create a model that has the learning of the 5?
Comments
Stats from TFA say 3 hours to fine tune on an A100 processor.
I wonder if one could "fine tune" specific layer values... That might be faster than updating every weight in every layer.
LoRA is an alternative to traditional fine tuning (which is usually done on specific layers as you mentioned).
To quote the LoRA paper[1]:
It's truly revolutionary: It basically lets you create a very small "diff" which you apply yo an existing model and it is suddenly fine tuned. These diff models are very small (5M for example).
[1] https://arxiv.org/abs/2106.09685
Or “multiplex” fine-tuning with inference, ie. do fine-tuning for 100ms, inference for 100ms, then tuning again… etc
Btw, is there a way to combine two or more models?
So for example, if I create 5 copies of a model, then fine-tune each copy with a different dataset -> can the 5 datasets be merged together somehow to create a model that has the learning of the 5?