200+ pages of Hugging Face secrets on how to train an LLM

Posted by eliebakk@reddit | LocalLLaMA | View on Reddit | 101 comments

200+ pages of Hugging Face secrets on how to train an LLM
Hey it's elie from the hugging face pre-training team! We're very excited to share our new blog (book?) that cover the full pipeline: pre-training, post-training and infra. 200+ pages of what worked, what didn’t, and how to make it run reliably :) [https://huggingface.co/spaces/HuggingFaceTB/smol-training-playbook](https://huggingface.co/spaces/HuggingFaceTB/smol-training-playbook) Hope yall will enjoy it, don't hesitate to make feedback on the community tab :)