Worth it to buy a modded 22GB 2080ti for LLMs?

Posted by define_undefine@reddit | LocalLLaMA | View on Reddit | 40 comments

I can get my hands on a 2080Ti with 22GB of VRAM instead of the original 11GB, for around $500/£400. Would you say this is worth it / the markup is fair given that unmodded variants can be had for slightly over half the price?

Right now I'm using A4000s which have been okay, but the 16GB limit is sometimes difficult to work around for running larger models. Also if buying the modded 2080Ti, I plan on getting two and connecting them using nvlink for better performance.

Given the emerging trend of better performing models running with less and less VRAM (phi, mixtral etc), one concern is buying this now and not needing it in a few months time.

Would appreciate any pros/cons that I might not have considered! Thanks