Gemma 4 Uncensored (autoresearch results)

Posted by adefa@reddit | LocalLLaMA | View on Reddit | 12 comments

Gemma 4 Uncensored — all 4 models, MoE expert abliteration, automated research loop

Released uncensored versions of all four Gemma 4 models. bf16 + GGUF for each.

Collection: https://huggingface.co/collections/TrevorJS/gemma-4-uncensored-69d2885d6e4fc0581f492698 Code: https://github.com/TrevorS/gemma-4-abliteration

Results

Model Baseline After KL Div
E2B (2.3B) 98% 0.4% 0.346
E4B (4.5B) 99% 0.7% 0.068
26B MoE 98% 0.7% 0.090
31B 100% 3.2% 0.124

Refusal rates from 686 prompts across 4 datasets (JailbreakBench, tulu-harmbench, NousResearch, mlabonne). Manually audited — most flagged refusals are actually the model complying with a disclaimer attached.

26B MoE

Standard abliteration only touches dense layers, which gets you from 98% → 29% on the MoE. The remaining refusals are in the expert weights. Used Expert-Granular Abliteration (EGA, concept from OBLITERATUS) with norm-preserving biprojection (grimjim) on each of the 128 expert slices per layer. That gets it to 3%.

How it was built

Set up an automated research loop — an AI agent reads the current results and idea backlog, picks the next experiment, runs it on the GPU, records results, and repeats. It ran 22 experiments across the 4 models, discovered the false-positive problem in standard refusal markers, built the cross-dataset evaluation, and implemented the MoE expert abliteration when dense-only wasn't enough.

Full experiment history and code in the repo.

Downloads

Each model has bf16 safetensors + GGUF (Q4_K_M, Q8_0):

Model bf16 GGUF
E2B link link
E4B link link
26B MoE link link
31B link link 
llama-server -hf TrevorJS/gemma-4-26B-A4B-it-uncensored-GGUF -c 8192

Apache 2.0.