Lora on top of lora-merged base model?

Posted by EntertainmentBroad43@reddit | LocalLLaMA | View on Reddit | 1 comments

My intuition is that it should be technically trivial after the merge, but not sure if anyone actually tried it. Has anyone tried lora- or qlora-ing on a previously lora-ed model? Maybe it’s possible but not recommended?

Reply to Post

1 Comments

[-]

disperaller@reddit

i did this by first training the lora adapter and then merged back to the original base model, call base model 1, then i trained another lora adapter from base model 1, then got the second trained lora adapter, merged back into base model 1, call it base model 2. test on both models, base model 1 shows significant improvement compare to original base model, however base model 2 seems to go wild, the score drops dramatically to nearly 0. No idea why this happens, the idea seems legit.