Great question! Its a 14+ year old socket and CPU setup perfect for what I need to run MySQL in RAG system, so I may need the cooling once I spin up my entire setup... hence the dual processors and 64GB memory also.
That said, $30 each fan and not many are available any longer that fit the socket. I received the fans prior to the main board and was jaw dropped by size. I was actually in a rush to get smaller ones, like a kid ready for Christmas morning. turns out the main board is an eATX and I could fit them like that. And yes it get hot
The CPUs are used in the coversions of the HFs to ggufs and does not even make a blip in htop and the GPU Is not used for that part.
I don’t think I will dual but will experiment. I have a larger interest in building parallel and RAG systems. Still working out the software to answer the rest. Yes that’s on carpet not a work desk. I used to build and know such importance, oh well.
it was built on a table mostly and a big plastic storage lid.Unfortunately it had to be there for a minute and got some pictures cause i forgot on the desk. It was a bit of logistics I had do.
oh this is interesting, the priority of GPU made the difference, cuda is choosing 3060 first. I flipped the CUDA_VISIBLE_DEVICES=1,0 and gained about 1/3 speed over just the 5060 on its own. so the 5060 then the 3060 and both I could see being used. "what is it about football that makes it so important to american society. Write a 3000 word essay describing the social events, financial gains, and other benefits. remove any negative vocabulary. be very positive."
3060 does bring down the || at least from a processing component. its about 2/3. I would need to see how much the VRAM can be used maybe without degradation but has the mem. I think I am mostly interested in having a RAG processing system that does not require model flipping. etc.
64GB being on an E5 2680 2400mhz, but the memory came from a mini-pc with 2 sodimms. I put them on an adapter until I can get a stack of memory chips. The memory is getting overclocked by the main board to 2600Mhz+. Th boot disk is for Ubuntu Server on a 240GB SSD. Then a Kingston 1024GB NVMe stick. This will also run MySQL where the DB Is about 10GB of data that is all a bunch of numeric data. I need the dual Xeon and high RAM GB.
eATX did not estimate the size and weight. The fans are overkill (or maybe not) they where on sale and only one of the few that fit the processor and main board now. I should have weighed it but at least 20 lbs I am getting.
Love it! I tried to do something similar with a 2690, 64gb system ram and two M40 with 48gb vram. But the motherboard couldn't map that much vram and it was a bust :/
Ouch. This has some specific design I had to make, I had to go with the Xeon E5 2680 v3 vs the V4 because my current machine I pulled the memory from is a sodimm into an adapter for that and no ecc memory.
Ideas really, I want to be able to have a system that can support those ideas. So this came together, dual Xeon E5 2680 about 2012. 64gb mostly for any RAG concepts. 3060 12gb and 5060 to 16gb
I have never trained and decided I had to get the two eGPU systems into one.
zilled@reddit
What's the advantage of the dual CPU in your scenario?
leonbollerup@reddit
qwuestion: why do you need that much cooling on the CPU on a AI rig ?
Frizzy-MacDrizzle@reddit (OP)
Great question! Its a 14+ year old socket and CPU setup perfect for what I need to run MySQL in RAG system, so I may need the cooling once I spin up my entire setup... hence the dual processors and 64GB memory also.
That said, $30 each fan and not many are available any longer that fit the socket. I received the fans prior to the main board and was jaw dropped by size. I was actually in a rush to get smaller ones, like a kid ready for Christmas morning. turns out the main board is an eATX and I could fit them like that. And yes it get hot
The CPUs are used in the coversions of the HFs to ggufs and does not even make a blip in htop and the GPU Is not used for that part.
leonbollerup@reddit
That’s actually fair .. bonus for that it cool.. 😎
FearFactory2904@reddit
https://i.redd.it/xpbklsq1ys1h1.gif
JustForFun-A@reddit
Hi ronald
Bulky-Priority6824@reddit
How much does the 3060 bring down the 5060? Have you tested single vs double
Frizzy-MacDrizzle@reddit (OP)
I don’t think I will dual but will experiment. I have a larger interest in building parallel and RAG systems. Still working out the software to answer the rest. Yes that’s on carpet not a work desk. I used to build and know such importance, oh well.
Bulky-Priority6824@reddit
It's that little 1 static zap you never know about and months later you start wondering why onboard lan not working, or random locks or bsod etc
I learned hard way 30 years ago
Frizzy-MacDrizzle@reddit (OP)
it was built on a table mostly and a big plastic storage lid.Unfortunately it had to be there for a minute and got some pictures cause i forgot on the desk. It was a bit of logistics I had do.
Frizzy-MacDrizzle@reddit (OP)
oh this is interesting, the priority of GPU made the difference, cuda is choosing 3060 first. I flipped the CUDA_VISIBLE_DEVICES=1,0 and gained about 1/3 speed over just the 5060 on its own. so the 5060 then the 3060 and both I could see being used. "what is it about football that makes it so important to american society. Write a 3000 word essay describing the social events, financial gains, and other benefits. remove any negative vocabulary. be very positive."
Frizzy-MacDrizzle@reddit (OP)
3060 does bring down the || at least from a processing component. its about 2/3. I would need to see how much the VRAM can be used maybe without degradation but has the mem. I think I am mostly interested in having a RAG processing system that does not require model flipping. etc.
Frizzy-MacDrizzle@reddit (OP)
ggml_cuda_init: found 2 CUDA devices (Total VRAM: 27761 MiB):
Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15849 MiB
Device 1: NVIDIA GeForce RTX 3060, compute capability 8.6, VMM: yes, VRAM: 11911 MiB
| model | size | params | backend | ngl | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
| llama 7B Q4_K - Medium | 4.07 GiB | 7.25 B | CUDA | 99 | pp512 | 2585.85 ± 32.42 |
| llama 7B Q4_K - Medium | 4.07 GiB | 7.25 B | CUDA | 99 | tg128 | 76.78 ± 0.05 |
xPXpanD@reddit
goodbye Ronald
it was good knowing you
Frizzy-MacDrizzle@reddit (OP)
He is in his home now and safe.
LLM_Contactee@reddit
Ronald looks muscular. How about RAM and disks?
Frizzy-MacDrizzle@reddit (OP)
64GB being on an E5 2680 2400mhz, but the memory came from a mini-pc with 2 sodimms. I put them on an adapter until I can get a stack of memory chips. The memory is getting overclocked by the main board to 2600Mhz+. Th boot disk is for Ubuntu Server on a 240GB SSD. Then a Kingston 1024GB NVMe stick. This will also run MySQL where the DB Is about 10GB of data that is all a bunch of numeric data. I need the dual Xeon and high RAM GB.
peligroso@reddit
That poor 5090 deserves a lot better.
CalligrapherFar7833@reddit
Ronald is a fat boy
Frizzy-MacDrizzle@reddit (OP)
eATX did not estimate the size and weight. The fans are overkill (or maybe not) they where on sale and only one of the few that fit the processor and main board now. I should have weighed it but at least 20 lbs I am getting.
CalligrapherFar7833@reddit
Congrats you have a 8 month old boy
Frizzy-MacDrizzle@reddit (OP)
haha
brickout@reddit
hi ronald
Frizzy-MacDrizzle@reddit (OP)
Firewormworks@reddit
Love it! I tried to do something similar with a 2690, 64gb system ram and two M40 with 48gb vram. But the motherboard couldn't map that much vram and it was a bust :/
Frizzy-MacDrizzle@reddit (OP)
Ouch. This has some specific design I had to make, I had to go with the Xeon E5 2680 v3 vs the V4 because my current machine I pulled the memory from is a sodimm into an adapter for that and no ecc memory.
FusionCow@reddit
hi ronald
Frizzy-MacDrizzle@reddit (OP)
Raygun
ComplexType568@reddit
hi ron
No_Run8812@reddit
Need more than that? What does it do? Config?
Frizzy-MacDrizzle@reddit (OP)
Ideas really, I want to be able to have a system that can support those ideas. So this came together, dual Xeon E5 2680 about 2012. 64gb mostly for any RAG concepts. 3060 12gb and 5060 to 16gb
I have never trained and decided I had to get the two eGPU systems into one.