Should I buy this RTX 2060 12GB graphics card at around $260 for AI purpose ?
Posted by Bharat01123@reddit | LocalLLaMA | View on Reddit | 19 comments
I’m interested in running Gemma 4 model/s for text only . It runs smooth even on my laptop but gets crazy hot.
Initially wanted to buy an 8 GB card. But I find this price for 12 GB good.
(Maybe I can run some image generation models too. But its not important.)
It has 6 Month Manufacturer warranty, and 2 years extended warranty for extra $22.
While RTX 3060 12 GB has almost double price.
Agitated_Chair_4977@reddit
gemma 3/4 text only on 12gb is fine. q4_k_m of the 12B fits comfortably, q6 of the 9B too. $260 with warranty is a fair price.
two gotchas: make sure its the actual 12gb sku (different from the 6gb launch card, easy to mix up), and check your psu - 184w tdp needs 8pin + 500w+.
honestly if you can stretch to a used 3060 12gb for like $50 more its way better. ampere kernels, lower idle power. worth a quick check on local market first
Bharat01123@reddit (OP)
Yes, I am using Q4 quant, and is perfectly fine. I might need to look into sku and psu.
huzbum@reddit
I wouldn't.
InsensitiveClown@reddit
No. You can get a used 4060Ti 16GB for that. The bare minimum I would touch these days is 40x0, and the 50x0 are already crucial for NVFP4.
anonymous_lurker-@reddit
Why not 30 series? Might be that the rest of the family is rubbish but the 3090 seems to be a go to recommendation. I'm fairly new to this
InsensitiveClown@reddit
NVFP8 support, latest features in CUDA 12.x for Ada architecture (SM8.9). Your mileage may vary. For me, my baseline is SM 8.9, and if I have to add new hardware these days, it'll be SM 9.0 at least, so, 50x0 cards due to the hardware support of NVFP4. This translates directly into VRAM savings. It'll be the difference between being able to run some models, or not. At all.
anonymous_lurker-@reddit
Can you give a practical example of what this means? I need to go do a bunch of research to work out what this all means. But what's something you can do on a 40 or 50 series card that doesn't work on a 3090 for example?
InsensitiveClown@reddit
Mate, I just told you...
Kahvana@reddit
Don't think I would, the 30 series and later are the first models to support BF16.
Bharat01123@reddit (OP)
Thank you, RTX 3060 8GB then. Not techie enough to deal with issues that might occur.
dampflokfreund@reddit
What? No. This will be a lot slower than the 2060 12 GB due to the lack of VRAM. There's not that much difference between a 3060 and a 2060 aside from BF16 support which doesn't matter for training. I'd say the 2060 12 GB is a good option if its that cheap for you.
Bharat01123@reddit (OP)
I have no idea how BF16 affects overall.
Antoniethebandit@reddit
There are plenty 3060 12 Gb for great price used
Bharat01123@reddit (OP)
Wanted to go for 12 GB because of the price. Otherwise 8GB is ok for my work.
And I dont know what to check before buying a used graphic card.
dampflokfreund@reddit
BF16 doesn't matter for local inference though. Only training.
Thin_Pollution8843@reddit
Nope. It’s old and architecture itself not supporting many modern features
dampflokfreund@reddit
Like what? Only BF16 comes to mind and that matters more for training.
totosse17@reddit
If you are up to old used hardware with outdated software stack I recommend looking at mi50
Agitated_Chair_4977@reddit
Gemma 3/4 text-only? 12GB is plenty headroom — Q4_K_M of Gemma 3 12B fits with room to spare, and even Q6 of the 9B is comfortable. $260 is fair if it's a known-good card with the 6-month warranty.
Two things to check before pulling the trigger:
Make sure it's the actual 12GB variant (RTX 2060 12GB is a separate SKU from the 6GB launch card — same name confusion bites people)
Power: 184W TDP, needs an 8-pin and a 500W+ PSU
For image gen as a side goal: SDXL is tight but SD 1.5 / SDXL Turbo run fine. Don't expect Flux speed.
Bigger picture though — if you can stretch to a used 3060 12GB for \~$50 more, you get Ampere (much better for inference with newer kernels) and lower idle power. Worth checking your local market.