What is MoRA? It seems to be an extension of LoRA.

[muooon]

MoRA: High-Rank Updating for Parameter-Efficient Fine-Tuning

MoRA has been something I've been curious about ever since I saw it before summer.
It may be more cost-effective than FFT.
If this can be applied to SD etc. with TrainTrain. I think in a dream.

Below are links to papers, etc.
https://arxiv.org/abs/2405.12130 (https://arxiv.org/html/2405.12130v1)
https://github.com/kongds/MoRA