Why is there no community project for training your own LLM from scratch on consumer hardware?
Posted by tevlon@reddit | LocalLLaMA | View on Reddit | 69 comments
ok so this has been bugging me for a while.
We've got nanoGPT/nanoChat from Karpathy which is honestly great and I'd point anyone to it. But here's the thing: to actually follow along and get real results you still end up renting cloud GPUs. And not everyone wants to drop $80+ on cloud compute just to mess around and learn. That barrier alone keeps a ton of curious people out imo.
So why isn't there a project (or even just a solid tutorial) built around one hard rule:
**it has to train on 8GB of VRAM. no cloud, no rented A100s.** if it doesn't fit on a normal gaming GPU it doesn't count.
The dream is a small but actually-real model trained on something like a Wikipedia dump, with a full writeup walking through the whole pipeline. And here's the part I really want: it should use the modern tricks people keep hyping but rarely bundle into one beginner-friendly thing. stuff like:
* BitNet / low-bit training to crush the memory footprint
* the Muon optimizer instead of plain old AdamW (apparently like 2x more compute efficient + decent memory savings, sounds perfect for a tight VRAM budget)
* aggressive quantization to stay inside 8GB
* whatever else helps squeeze a trainable model onto consumer hardware
basically nanoGPT's vibe but with a hard "must run on your gaming PC" constraint and a modern technique stack, so anyone can train a model end to end for free.
so my questions:
1. does this already exist and I just haven't found it? if so please link
2. if not... anyone wanna build it together?
69 Comments
Gunnarz699@reddit
tevlon@reddit (OP)
Gunnarz699@reddit
Silver-Champion-4846@reddit
Gunnarz699@reddit
Silver-Champion-4846@reddit
Gunnarz699@reddit
Silver-Champion-4846@reddit
hopbel@reddit
Formal-Exam-8767@reddit
UnethicalExperiments@reddit
unknowntoman-1@reddit
darkpigvirus@reddit
Hopeful_Ad6629@reddit
Voxandr@reddit
Aromatic-Low-4578@reddit
CubicleHermit@reddit
NicolaZanarini533@reddit
carloselieser@reddit
stiflers-m0m@reddit
tevlon@reddit (OP)
Gunnarz699@reddit
FreQRiDeR@reddit
dash_bro@reddit
stonetriangles@reddit
tevlon@reddit (OP)
Polite_Jello_377@reddit
dinerburgeryum@reddit
Snoo_81913@reddit
tevlon@reddit (OP)
Snoo_81913@reddit
tevlon@reddit (OP)
Snoo_81913@reddit
tevlon@reddit (OP)
Snoo_81913@reddit
tevlon@reddit (OP)
llama-impersonator@reddit
Snoo_81913@reddit
Snoo_81913@reddit
giveen@reddit
tevlon@reddit (OP)
llama-impersonator@reddit
giveen@reddit
llama-impersonator@reddit
BidWestern1056@reddit
iMrParker@reddit
tevlon@reddit (OP)
Such_Advantage_6949@reddit
tevlon@reddit (OP)
Such_Advantage_6949@reddit
iMrParker@reddit
fulgencio_batista@reddit
Double_Cause4609@reddit
tevlon@reddit (OP)
stonetriangles@reddit
tevlon@reddit (OP)
Double_Cause4609@reddit
dinerburgeryum@reddit
BoogerheadCult@reddit
Valuable_Touch5670@reddit
tevlon@reddit (OP)
CreamPitiful4295@reddit
andy_potato@reddit
tevlon@reddit (OP)
galibert@reddit
tevlon@reddit (OP)
ketosoy@reddit
Lissanro@reddit
AustinSpartan@reddit