Running 4x V100S-PCIE-32GB in a Dell T640 tower, so I can speak to this directly.

You said homelab and that was the key word in my decision too. I specifically went PCIe cards in a used server chassis over SXM2 on adapter boards. SXM2 is cheaper per card but your dealing with adapter boards, cooling headaches, and a build that's harder to maintain. PCIe in a proper server with hot-swap fans, iDRAC remote management, and standard power delivery just made more sense for something that lives in my house and needs to stay running. The T640 shows up on eBay regularly and the GPU cage fits 4 full-length cards without any fabrication.

All-in I'm under $6k for 128GB of VRAM across 4 cards, including the server, RAM, CPU upgrades, and all the GPU hardware. When I was shopping, 3090s were going for $1100-1200 each and that gets you 3 cards at 72GB total for the same money. The math just didn't work for what I wanted, which was enough VRAM to run multiple large models simultaneously.

The purpose of my build is a local voice assistant, STT, LLM inference, TTS, the whole pipeline running on the tower. Right now it runs 95-100% local. I haven't hit a cloud API in weeks. I've got a 35B MoE on 1 as the primary brain, a 31B dense model on number 2 for tool routing, and GPUs 3+4 running a flex pool that swaps between a 120B, an 80B, and a 70B model depending on the task. Thats 3 models hot, 2 more on standby, all local.

Software stack: llama.cpp is the daily driver and it works great on Volta with no issues. I also tested the 1Cat-vLLM fork which is specifically patched for V100, got 121 t/s single user and 502 t/s at concurrency 8 on a 35B MoE. vLLM wins for concurrent workloads but there's a big caveat: you MUST run with enforce-eager disabled or performance tanks by 7x. Standard vLLM without the 1Cat patches won't run on Volta at all in recent versions.

For the flex pool on GPUs 2+3, the killer feature is how easy GGUF models are to work with. I download a model, drop it in a directory, and llama.cpp just loads it. I've got 320GB of system RAM so I pin the model files in page cache, swapping between a 60GB and a 46GB model takes about 26 seconds from RAM vs 90+ seconds from NVMe. For experimentation, being able to rotate through models in under 30 seconds without touching config files is huge.

Honest negatives: no BF16, no FP8, and parts of the ecosystem are dropping sm_70 support. TensorRT-LLM and TGI won't work. vLLM only works with the community fork. If your planning to use anything outside of llama.cpp, verify Volta support before buying. Power draw is real, I'm currently running at default clocks and the cards pull about 200W each under load. I've got 2000W PSUs arriving so I can test uncapped performance, but right now I'm bandwidth-bound on generation anyway so more power doesn't necessarily mean more speed.

To directly answer your question: yes, 32GB V100s absolutely still make sense for homelab AI in 2026, but only if you go in with the right expectations. They're not fast in absolute terms, a 5090 will smoke them on per-card throughput. What they give you is raw VRAM density at a price point nothing else touches. If your goal is running larger models locally and experimenting with different architectures, that VRAM is what matters. If your goal is maximum speed on a single small model, buy the newer card.