Crazy idea: Derestricted Llama 405B?

Posted by My_Unbiased_Opinion@reddit | LocalLLaMA | View on Reddit | 0 comments

I've been looking at the larger MOE Derestricted models and notice the bigger models seem to improve when derestricted. (UGI benchmark) I feel like 405B has a ton of capability that might be locked behind its safety tuning. 405 is the most dense model large model we have right now and I have a hunch it can improve a ton of we let it's off it's leash. This obviously would need a lot of hardware to do. But I'm wondering if anyone thinks this model would shine derestricted as well.