My hardware hasn't changed much, and probably won't until after RAMageddon is over, however many years that takes.

I have multiple dual-processor Xeon servers, with a mix of E5-2660v3, E5-2680v3, and E5-2690v4. Most have 256GB DDR4-2133, but two have 128GB DDR4-2133.

One has a 32GB MI50 which keeps Big-Tiger-Gemma-27B-v3 resident, another has a 32GB MI60 which keeps Gemma-4-31B-it resident, both quantized to Q4_K_M.

Mostly those Xeons are busy running GEANT4 and Rocstar simulations (which is what I originally bought them for), but I'll also use them to infer with GLM-4.5-Air or K2-V2-Instruct via pure-CPU (with llama-completion from llama.cpp), so that they aren't evicting the models from my GPUs (which wouldn't speed up these large models much anyway).

I also have some odds and ends, including a Dell T7500 with a E5504 Xeon and 24GB of DDR3, with a second PSU literally duct-taped to it and daisy-chained via ADD2PSU device, to feed my third GPU, a 16GB V340. That V340 keeps Qwen3.5-9B Q4_K_M resident.

My main use-cases for these models are:

• GLM-4.5-Air: Non-agentic codegen, physics assistant (mostly critiquing my neutron transport notes and suggesting relevant subjects for further study), and medical assistant (mostly explaining medical journal publications to me).

• Gemma-4-31B-it: Wikipedia-backed RAG for general Q&A, creative writing, business writing, language translation, Evol-Instruct pipelines, and sometimes debugger for GLM-4.5-Air's code. Also working toward using it for a technical support IRC chatbot for a channel I moderate.

• Big-Tiger-Gemma-27B-v3: Critiques my Reddit activity and provides constructive criticism, also great for persuasion research and violent creative writing (Murderbot Diary fan-fic; non-erotic but very violent). I'm looking forward to TheDrummer giving Gemma-4-31B-it the Big Tiger treatment so it can take over these tasks.

• K2-V2-Instruct: Long-context tasks like system log analysis and IRC log analysis, also what my "actlikettk" (self-clone) script uses, though Gemma4 might be taking over that role, not sure yet. It is also very good at Linux technical assistance and cybersecurity. I'd like to use this model for more kinds of tasks, since it's wicked smart and reasonably up-to-date, but it's hellaciously slow on my hardware (0.5 tokens/second at short context, slowing to 0.06 tokens/second at 227K context), so that will need to wait until I have at least 128GB of VRAM to throw at it.

• Qwen3.5-9B: Synthetic dataset upcycling and augmentation. The dataset upcycling is mostly following what https://arxiv.org/abs/2510.10681 describes but with low-quality synthetic data instead of web data, and the augmentation is mostly replicating and refining the techniques LLM360 used to generate their TxT360_QA dataset.

When TheDrummer publishes a new Big Tiger based on Gemma 4, that will likely subsume the roles of both Gemma-4-31B-it and Big-Tiger-Gemma-27B-v3, which will leave one of my 32GB GPUs empty. I haven't decided what to do with it, yet. Maybe Olmo-3.1-32B-Instruct to synthesize an ontological syllogism corpus? I've been neglecting that project for a while now, and probably need to rethink it. Modern "reasoning" techniques have gone a different way.

The GPU-resident models are hosted via llama.cpp's llama-server, and I use them via its API endpoints with Python/Perl scripts. The pure-CPU inference models are run via bash scripts which wrap llama.cpp's llama-completion.