New 31M and 14M Pythia models???!!!
Posted by Ok-Type-7663@reddit | LocalLLaMA | View on Reddit | 5 comments
[yeah pythia is quite old but - new models since february???!!!]()
Posted by Ok-Type-7663@reddit | LocalLLaMA | View on Reddit | 5 comments
[yeah pythia is quite old but - new models since february???!!!]()
silenceimpaired@reddit
31m? That's insanely small. I'll be surprised if it can create a sentence.
power97992@reddit
Even a 2M param model can produce simple sentences
silenceimpaired@reddit
Sure, autocorrect and autocomplete might benefit from it... But I find even a 3b model is incapable of creating a reasonably deep story.
AnonymousTransfem@reddit
Pythias are really nice base models since theyre just trained on The Pile , from 2020
and so theres no AI inbreeding and its way easier to avoid the LLM-speak
DeltaSqueezer@reddit
I'm glad EleutherAI is still going!