Struggling to Host & Fine-Tune LLaMA 3.2 3B on the Cloud - Any Tips?
Posted by Necessary_Round8009@reddit | LocalLLaMA | View on Reddit | 8 comments
Hey everyone,
I’ve been trying to host and fine-tune LLaMA 3.2 3B on a cloud platform like Lambda Labs, but honestly, it’s been pretty tough. Setting everything up and getting it to run smoothly feels more complicated than I expected.
I was wondering if anyone here has found an easier way to handle this? Maybe a more beginner-friendly method or tools that can simplify the process? I’d love to hear how you’ve managed to make it work!
Thanks so much for any advice you can share!
reza2kn@reddit
Enter Unsloth:
https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Llama3.2_(1B_and_3B)-Conversational.ipynb
You can fine-tune Llama 3.2B 1B and 3B models here on Colab for free ;)
Necessary_Round8009@reddit (OP)
Hoo thanks I will give it a look. But if I understand my file are local but I only use the GPU from the cloud ?
reza2kn@reddit
No. If you have your files on your hard drive right now, you'll first have to upload them somewhere, which could be HuggingFace, or even Google Drive itself, and then yes, you'll only use the T4 TPU from Google that'll give you 16GB of VRAM and should be more than enough 🤘🏻
Necessary_Round8009@reddit (OP)
Did you have a good tutorial ? I gave it some try but still got some errors
reza2kn@reddit
Try taking a look at the UnSloth docs. What errors did you run into? Did you ask Sonnet about them?
ForsookComparison@reddit
What method are you using and where do you get stuck?
Lambda Labs is pretty much "click the button, here's your ssh key, have fun" in terms of infra setup. Are you stuck on installing Llama CPP or similar?
Necessary_Round8009@reddit (OP)
I’m downloading LLaMA directly from Meta's site via SSH. After that, ChatGPT mentioned I need to convert it into a Hugging Face model, but I keep running into an issue. It gives me an error saying the tokenizer is corrupted.
ForsookComparison@reddit
ChatGPT and Claude are both weirdly bad at fine tuning llama. Likely a combination of how quickly and recently this has all changed and that it's against both their best interests lol. I had the exact same problem.
I recommend finding a good old fashioned blog post or YouTube video that teaches you start to finish.