Where can I get A100 gpu for free or cheap
Posted by Optimal-Resident694@reddit | LocalLLaMA | View on Reddit | 22 comments
I’m an undergrad student working on an llm hybrid training optimization for a research paper. I’m using the mistral7b model and hitting ram limits in colab. I need the A100 gpu as it provides more ram and also to implement the flash attention hardware optimization. I would really appreciate any advice.
carl_peterson1@reddit
Co-founder of Thunder Compute here, many of our users are in this same situation where they hit colab limits and check us out, A100s start at $0.78/hr. If you sign up with your student email you'll get $20 of free credit.
True-House6584@reddit
i signed up with my student mail but i didnt get the 20 dollars...
Carl_ThunderCompute@reddit
Shoot our support an email or message in our Discord, can take a look at your account there. Two common causes: 1) some countries aren't currently serviced or 2) the discount only applies to accredited universities, if this is a high school etc. email you won't receive credit.
agrochipsy@reddit
what is the catch, how so cheap?
Carl_ThunderCompute@reddit
We spend our time building low level software to orchestrate GPUs more efficiently, which lets us pass on lower prices to you. Let me know your thoughts if you end up testing it out, I’m here to help
No_Surprise8876@reddit
Hi, I’m interested in your GPU server setup. Could you clarify the connection workflow? Specifically, do you support direct remote connections via VS Code (Remote-SSH), or are users required to submit tasks through a scheduler like Slurm?
Carl_ThunderCompute@reddit
Short answer: yes, you can connect via standard SSH.
Long answer: we also provide our own CLI and VS Code extension which wrap remote SSH or standard SSH to manage keys for you and make it simpler to connect. The simplicity here is a large part of why people choose our platform.
pharrowking@reddit
i see you're using collab, but is it paid? the monthly subscription for collab, gives you an a100 40GB card
Optimal-Resident694@reddit (OP)
i want to purchase colab pro, but it says online that a100 is not guaranteed
Puzzled-Truck9932@reddit
it's usually available tho
spindoc-fe@reddit
Would $1,000 be a fair price for an Nvidia Tesla A100 80GB GPU?
There are currently 8 of them up for auction. While there are fees on top of that, it might still be a good deal.
Jordanquake@reddit
Co-founder of Thunder Compute here, many of our users are in this same situation where they hit colab limits and check us out. If you sign up with your student email you'll get $20 of free credit.
Lucaspittol@reddit
A100 on colab sometimes come with 80GB and cost about 7 compute units per hour. A $10 monthly subscription gets you 100 compute units, so do the math.
Expensive-Paint-9490@reddit
Modal.com
vast.ai
Mkengine@reddit
Maybe this helps?
ashersullivan@reddit
Check if your institution ha any academic partnership with cloud privders like many schools have GCP or AWS research credits... also look into lambda labs student program, paperspace gradient or app;y fr credit throug hugging face bigscience program. For pure research work, kaggle notebbooks offer free tesla P100s which might be enough fo mistral 7b if you optimize memory usage. Not A100s but workable for prototyping before scaling up
exaknight21@reddit
I dont know how credible but there are chinese cloud rent services that are very cheap. I forget off top my head but you can ask chatgpt.
Prestigious_Thing797@reddit
Search up cloud providers and shop around. I'd anticipate prices roughly around $1.50/hr per gpu but I haven't looked recently.
Optimal-Resident694@reddit (OP)
i checked google cloud and it would be approx 40 usd to complete my project so im just wondering if theres a cheaper option
Prestigious_Thing797@reddit
I would reach out to folks in the CS department at your college and see if there's any compute they can share or any ways the school could provide funding.
Optimal-Resident694@reddit (OP)
oh yeah, thanks
Snoo_64233@reddit
Just apply to work at McDonald for 2 hours. Then quit. They pay $20 per hour.