Newbie needs a recommendations

Posted by StationNo5516@reddit | LocalLLaMA | View on Reddit | 5 comments

Hey guys Im totally new to local LLMs overall but I have great experience with ai automation and backends overall all using the gemini api I wanna try to work with the new Gemma 4 its quite impressive tbh it won’t be working for coding (until I buy a new gpu) I don’t care about response time all I care about is the accuracy and output quality overall it can work for the whole day for two tasks its ok I will connect it to openclaw so what model do you think will be more suitable for this work and my pc can run

2070 Super 8GB

32 giga ram

Ryzen 7 3700X

And Im thinking to buy a 6800XT 16giga vram

I will keep the 2070 super as personal and the rx will be for the llm and openclaw but I can’t upgrade more again for years

But Im scared that AMD can be not compatible with some models if I wanted to try is this true?

Thanks

[-]

ai_guy_nerd@reddit

RTX 6800 XT is solid for local model inference, way better than the 2070 Super. You'll get good compatibility with most frameworks (Ollama, vLLM, LM Studio all support AMD well at this point). One thing though: if you're running OpenClaw + a local LLM together on the same box, watch your VRAM usage. A 16GB card handles most things up to 34B models comfortably, but if you're running two heavier processes in parallel, it gets tight.

For what you're describing, skip Gemma 4 unless accuracy on that specific task matters more than anything else. Qwen 3.5 variants hit much better quality-to-size ratios. The 32GB RAM + 3700X will handle the CPU overhead fine.

One heads up: AMD driver updates on Linux can be flaky, so pin your ROCm version once you get a working setup. The GPU itself is solid though.

Fun_Librarian_7699@reddit

I think Gemma E4B will be fast on your system, but it's maybe too small for tasks like openclaw.

StationNo5516@reddit (OP)

Can’t my device handle the MoE model with low response rate like 10 or 15 t/s?

It will even handle the gemma-4-31B at low speeds

Status_Record_1839@reddit

RX 6800 XT works fine with llama.cpp via ROCm or Vulkan — compatibility isn't really a concern for inference. With 16GB VRAM you can run Qwen2.5-14B Q4 comfortably, which is solid for tasks like openclaw workflows.