Comment on Vera: Vector-Based Random Matrix AdaptationComments−egnehotsOP2yVeRA makes LoRA ~10x more parameter efficient while retaining the same performance.It's somewhat like a recursive LoRA scheme, where the LoRA A and B matrices are also decomposed using two small trainable vector parameters.
Comments
VeRA makes LoRA ~10x more parameter efficient while retaining the same performance.
It's somewhat like a recursive LoRA scheme, where the LoRA A and B matrices are also decomposed using two small trainable vector parameters.