I don't recall how I found out about it, but it was either paperswithcode or an LLM research session working through the intruder dimensions problem.
In my Stable Diffusion tests, it substantially improves LoRA training speed and fidelity, though I've got some experiments that seem to even further substantially improve on it by adding learnable rotations of the singular vectors.
Comments
The one I was referring to was from this paper, first published in May: https://arxiv.org/abs/2505.20211v1
I don't recall how I found out about it, but it was either paperswithcode or an LLM research session working through the intruder dimensions problem.
In my Stable Diffusion tests, it substantially improves LoRA training speed and fidelity, though I've got some experiments that seem to even further substantially improve on it by adding learnable rotations of the singular vectors.