Joined the 48GB Vram Dual Hairdryer club. Frankly a bit of disappointment, deepseek-r1:70b works fine, qwen2.5:72b seems to be too big still. The 32b models apparently provide almost the same code quality and for general questions the online big LLMs are better. Meh.

You appear to be downloading non-quantized versions. Are you using the ollama site? Don't. Get on huggingface and look up quantized versions of the models you want. Use their Local LLM Leaderboard to scope out models here - https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/ Stick to 'Only Official Providers' until you know what you're doing, as a lot of the custom tuned models are centered entirely around achieving the best benchmark scores, and make sacrifices in other areas. You can get Q4 versions that are like 1/8th the size of the full f32, and sacrifice little accuracy. Or a Q6 is I think 1-3% precision loss, or something like that, and they're **5x smaller**. The upshot is you can fit much better models entirely in vram. With your 48gb, i'd be trying out quants of 70-72B models. You can even quantize your context, at basically no precision loss, but that's a different topic. To download quantized models on huggingface, find the model you want, and go to its page. Click 'Quantizations' (on the right). Sort those by most downloaded. Click the top downloaded GGUF version. You can click on the stuff on the right like Q4_K_M, or Q6_K, and you'll get a tab pop out with the size. When you find the size you like,click 'Use This Model' at the top of the popout, then click the Ollama button. Copy the link, open a command prompt and paste it in. It'll download it for you, and then run it in the command prompt window. Here's a link for a Qwen2.5-32B, which is one of the top performing models. This one is at at Q6_K. Paste it into command prompt if wanna give it a go: ollama run hf.co/bartowski/Qwen2.5-32B-Instruct-GGUF:Q6_K Hope this helps, and wish this was the first thing I read 😆.