Is there a version control system similar to Git but for training states of LLM
Posted by labratdream@reddit | LocalLLaMA | View on Reddit | 1 comments
Basically it should help easily commit, track, reverse model training states across members if training team. I guess it may require massive disk space in case of larger models.
If such softwate doesn't exist how do you guys store and manage training states of models ? What would be the most efficient database type to store training states ?
ElectricDilbert@reddit
You can use Diversion for that, it's similar to git in many ways, but supports files of virtually unlimited size and is much faster than Git LFS (Disclosure - I'm one of the founders)