axolotl vs unsloth [performance and everything]

Posted by Shivacious@reddit | LocalLLaMA | View on Reddit | 26 comments

there has been updates like (https://github.com/axolotl-ai-cloud/axolotl/releases/tag/v0.12.0 shoutout to great work by axolotl team) i was wondering ,is unsloth mostly used for those who have gpu vram limitations or do you guys have exp is using these in production , i would love to know feedback from startups too that have decided to use either has their backend for tuning, the last reviews and all i found were 1-2 years old. they both have got massive updates since back than

Reply to Post

26 Comments

[-]

MR_-_501@reddit

Good experience with axolotl personally, but more from a production perspective than hobby necessarily, the config yaml workflow makes it easy to evaluate how different models adapt to a certain dataset for example. If you use a pytorch-dev docker container it works great, their own containers are broken. Unsloth works well but you cán get the same memory efficiëncy in axolotl by enabling lora optimizations, the reason that it is not on by default is that it does negatively impact the performance of the final model, but you wont notice this most of the time, especially when you do not have a huge dataset. The colab notebooks that unsloth provides are great though, and give a great introduction to finetuning. So i would actually recommend it more if you are just starting out.

[-]

Shivacious@reddit (OP)

absolutely. that's what confused me the most when i checked that axolotl provides the lora stuff and all (https://docs.axolotl.ai/docs/lora\_optims.html) and i was sort of confused that they moved away from using unsloth and did their own implementation

[-]

MR_-_501@reddit

Axolotl existed before unsloth, its all open source and they use each others innovations. Just a different method of usage from a user perspective.

[-]

yoracale@reddit

What exactly do we use from Axolotl? We don't use any code from Axolotl whatsoever?

[-]

FullOf_Bad_Ideas@reddit

I've used Axolotl first, then switched to Unsloth when it released. Then I switched to Llama-Factory when I went pro. As an example of differences. Unsloth managed to get GPT-OSS 20B quantizable to NF4, so you can finetune it in FREE Google Colab. For 120B you need single H100. Axolotl has example where you can finetune, with LoRA, 20B on 48GB of VRAM, and 120B on 8xH100 setup (though I guess 2xH100 could be enough there). As a hobbyist, I'd prefer Unsloth, since I can actually do something interesting locally. Unsloth is great for local experimenting on my own hardware, I couldn't do professional finetune with it due to messy behaviour at one time which would require me to have like 1TB of RAM to make a finetune (I don't think I can get into specifics), Axolotl was painful to use when I wanted to work professionally on it, and Llama-Factory is great when I want to work professionally on finetuning models. I hardly have time to go back to trying out software that I got burned on, so Axolotl might be great, but I don't know if I'll spend significant time on it. I don't think I am the only one burned on Axolotl, a lot of thinks were buggy like 6-12 months ago, people moved on to different projects I think.

[-]

CyberNativeAI@reddit

Axolotl examples for gpt-oss are full fine-tune tho

[-]

FullOf_Bad_Ideas@reddit

no, that's a not full finetune ># LoRA SFT linear layers (1x48GB @ ~44GiB) axolotl train examples/gpt-oss/gpt-oss-20b-sft-lora-singlegpu.yaml For Unsloth, the requirements to run LoRA of GPT OSS 20B is free Google Colab, because they were able to get QLoRA running.

[-]

CyberNativeAI@reddit

I guess it’s easier for me to rent few b200 for few hours, unsloth is great but single GPU

[-]

yoracale@reddit

multiGPU already works with unsloth we just haven't officially it yet but we're making it much better/easier fyi: https://docs.unsloth.ai/basics/multi-gpu-training-with-unsloth

[-]

CyberNativeAI@reddit

IDK looks pretty good to me: # LoRA SFT linear layers (1x48GB @ ~44GiB) axolotl train examples/gpt-oss/gpt-oss-20b-sft-lora-singlegpu.yaml # FFT SFT with offloading (2x24GB @ ~21GiB/GPU) axolotl train examples/gpt-oss/gpt-oss-20b-fft-fsdp2-offload.yaml # FFT SFT (8x48GB @ ~36GiB/GPU or 4x80GB @ ~46GiB/GPU) axolotl train examples/gpt-oss/gpt-oss-20b-fft-fsdp2.yaml

[-]

EconomicMajority@reddit

What’s good about llama factory in your opinion?

[-]

FullOf_Bad_Ideas@reddit

Pretty much bug free, stable, with a lot of models that I care about supported there (I tend to be interested in non-Western models more by the nature of their performance so I don't care about LiquidLM or Llama 4 much) and appropriate number of things to switch around to get good performance out of the final model.

[-]

EconomicMajority@reddit

Thanks. I found it a bit unintuitive to get started but it seems pretty good. Especially in terms of modes/model support.

[-]

toothpastespiders@reddit

I think a lot just comes down to what you're used to. They both have a lot of quirks and familiarity with that is often as important as the nature of the framework itself. That said, the lack of multi gpu support on the free version is a big limiting factor with unsloth.

[-]

stoppableDissolution@reddit

It does unofficially (but with endorsement from devs) work with accelerate on free version for quite a while, and official support comes soon.

[-]

yoracale@reddit

Yes correct! We're trying our best to pump it out but new model releases lag is behind

[-]

yoracale@reddit

multiGPU already works with unsloth we just haven't officially it yet but we're making it much better/easier fyi: https://docs.unsloth.ai/basics/multi-gpu-training-with-unsloth

[-]

iamMess@reddit

Axolotl is currently the fastest framework. It takes a little more to set up, but it’s still really easy to use.

[-]

Evening_Ad6637@reddit

I once tried to finetune with axolotl - it didn’t work, it crashed with some python errors and I was too lazy to fix it. Then I tried it with unsloth and it worked perfectly. I love unsloth's notebooks since they are also very educational. After 30 minutes, I had a small llama model that knew my name and who I was, etc.

[-]

EconomicMajority@reddit

Axolotl seems to use like 2x more vram than it needs to. I use qlora-pipe even tho it’s basically abandoned at this point because it’s the only thing that lets me do fine tuning on multi gpu with decent parameters without running out of vram.

[-]

Watchguyraffle1@reddit

Same. I never got axolotl working

[-]

Shivacious@reddit (OP)

interesting. what gpu were you using ? do you remember the error ? possibly it could have been the size of input too

[-]

Evening_Ad6637@reddit

I have Rtx 3090 Ti and I trained llama 3b or 1b, can’t remember exactly. Unfortunately I can’t remember what the error message was.

[-]

terminoid_@reddit

GRPO memory requirements are pretty insane without unsloth imo

[-]

Watchguyraffle1@reddit

Same. Never got axolotl working

[-]

llama-impersonator@reddit

axolotl being config rather than code based makes it easier to replicate training runs and extend it to different datasets or mess with the hparams while not worrying about maintaining a bunch of tuning scripts with minor alterations or a fat stack of command line options.