New 31M and 14M Pythia models???!!!

[yeah pythia is quite old but - new models since february???!!!](

[-]

31m? That's insanely small. I'll be surprised if it can create a sentence.

[-]

Even a 2M param model can produce simple sentences

[-]

Sure, autocorrect and autocomplete might benefit from it... But I find even a 3b model is incapable of creating a reasonably deep story.

[-]

Pythias are really nice base models since theyre just trained on The Pile , from 2020

and so theres no AI inbreeding and its way easier to avoid the LLM-speak

[-]

I'm glad EleutherAI is still going!