Crazy idea: Derestricted Llama 405B?
Posted by My_Unbiased_Opinion@reddit | LocalLLaMA | View on Reddit | 0 comments
I've been looking at the larger MOE Derestricted models and notice the bigger models seem to improve when derestricted. (UGI benchmark)
I feel like 405B has a ton of capability that might be locked behind its safety tuning. 405 is the most dense model large model we have right now and I have a hunch it can improve a ton of we let it's off it's leash.
This obviously would need a lot of hardware to do. But I'm wondering if anyone thinks this model would shine derestricted as well.
0 Comments