Will I have any problems pairing a 3090 with a 5060 Ti 16GB?
Posted by PhantomWolf83@reddit | LocalLLaMA | View on Reddit | 5 comments
I've been wondering how feasible would it be to have a dual GPU setup of a 3090 and 5060 Ti 16GB compared to two 5060 Tis. I plan to use the 3090 for LLMs for the higher bandwidth and token generation, and the 5060 Ti as my primary and gaming GPU for the lower power consumption and temperatures and more modern feature set. If I need to I can combine the VRAM for 40GB.
Will there be any compatibility or any other problems with this configuration when using them together for bigger models (I mostly use KoboldCpp, not sure about other LLM programs)? Also, the speed is definitely going to be slower, but how much slower? Will it drop to the speed of the slower card (5060 Ti) or the average of the two?
Evening_Ad6637@reddit
No, it's absolutely no problem to combine the two GPUs.
And no, using the 3090 as a second card instead of a 5060ti will not negatively affect performance. The 3090 will actually make things faster, as it has a bandwidth of ~950 GB/s, which is almost twice as fast as the 5060ti.
This means that when running LLMs, you should use the 3090 first and only add the 5060ti if you need more than 24 GB.
As for average bandwidth, you need to consider how many layers have been offloaded to which GPU. Since the 3090 accounts for 60% of the maximum combined 40 GB, your worst-case bandwidth would be (9500.6) + (4500.4) = 750 GB/s.
AppearanceHeavy6724@reddit
Weighted average of two ((24930+16450) / 40) will be you real bandwidth if model does not fit jnto 3090 and full 3090 speed if it fits into it.
Smooth-Cow9084@reddit
I have those 2 and on ollama got only 10% less speed when running olmo 3 32gb. But I believe vllm causes trouble if splitting between different models
MitsotakiShogun@reddit
It will run fine. I had a 5070 Ti + 3090 setup for a few months (Windows / WSL) and I didn't have any compatibility issues in any of the apps or frameworks/libraries I used.
PhantomWolf83@reddit (OP)
Thanks, good to know!