15 years from, power like this will likely be running on your laptop. 15 years ago it would probably have ranked in the global top 500 supercomputers list.

[-]

Pazzeh@reddit

I would be shocked if that turns out to be true

[-]

There was a 6 year gap between 690 and 3090, and 3090 is a little over 4 times as powerful as 690. I don't think we will have laptop with the power of 15 x 3090 in 11 to 15 years from now. 4090 is only 76% more powerful than 3090 (with same VRAM), and the upcoming 5090 will have a similar boost in performance (or lower) with only slightly better VRAM. That's 3 x performance jump (at most) in 4 years.

[-]

zyeborm@reddit

You'll probably find dedicated AI hardware instead of GPUs by then. They will have a lot more performance and lower power consumption due to architectural changes. Personally I think mixed memory and pipelined compute will be the kicker for it.

[-]

novus_nl@reddit

That's actually pretty interesting, like have a dedicated GPU for visual rendering AND a AIPU for generating/calculating AI output.

she said, " let them eat brioche" if I recall Bill said 256k ought to be enough ...

[-]

kernald31@reddit

Rousseau wrote "Qu'ils mangent de la brioche" in 1767, Marie Antoinette was born in 1755. It's frankly quite unlikely that she was the princess Rousseau was talking about.

Bill said 256k ought to be enough

No. Bill Gates, like any other software engineer back then, would never have said anything of the sort - quite the opposite. Managing to get a computer running with an address space large enough to handle 640kB of memory was quite a feat - and a very welcome one. The quote (wrongfully) attributed to him is about 640kB, not 256.

[-]

Reasonable_War_1431@reddit

my post says 640 - not 256 ? It was IBM who was quoted on this . Gates merely agreed

perhaps the quote attributed to Marie was her recap of Rousseau's remark as a double entendre - that would exhibit the former Austrian Royal's learned reads of french authors as she was being groomed by the court's best educators to speak and read french - it was very important for the new queen to become a francophile to show les peuples she had embraced her country and its customs as their new monarch. Birthday May 16th - same day as Jean D 'Arc and Nero - they all certainly lead extreme lives

[-]

kernald31@reddit

Regardless of the amount - Bill Gates never said anything of the sort nor agreed. No software engineer at the time would have ever said anything of the sort when they were all struggling to enable more memory to be used.

Regarding Marie Antoinette, her repeating something she's read hardly make it a quote by her.

[-]

Reasonable_War_1431@reddit

she had more ears listening to her mouth than eyes reading Rousseau - That would be why she was cast as the author if that quote

[-]

soytuamigo@reddit

Yep, unlike

The world today has 6.8 billion people. That's headed up to about nine billion. Now, if we do a really great job on new vaccines, health care, reproductive health services, we could lower that by, perhaps, 10 or 15 percent. But there, we see an increase of about 1.3.

thrownawaymane@reddit

Isn't part of the problem relativistic effects at those sizes? That won't go away with 3d stacks.

[-]

justintime777777@reddit

Are you thinking of Quantum effects? (like quantum tunneling, where electrons jump through Gates and Channel they classically shouldn’t)

Say you made a 10 layer chip with 10nm transistors,
You wouldn't get any quantum tunneling like you would at 1nm, but you would get transistor density equivalent of 1nm.

Stacking is complicated and does hurt thermals on the inner layers, but with 1nm equivalent tech you could run things slower and more efficiently to compensate.

[-]

Odd-Interaction-453@reddit

It still is for Linus Torvalds, just saying, lol.

[-]

kalloritis@reddit

And each will require its own 1600W PSU

[-]

zuilserip@reddit

Each RTX3090 can reach 35.5TFlops (fp32), so 15 of them would get you to around 530TFlops. This should get you into the Top 500 list as recently as 2016, and get you to the top of the list in 2007, duking it out with IBM's 212,992 processor BlueGene/L monster.

15 years ago (in 2009), a single RTX3090 would get you into the list.

[-]

masterlafontaine@reddit

Only if we leave silicon behind

[-]

J-IP@reddit

2009 the gtx 295 had 896 MB of accesible vram because of cut down memory buss but lets call it 1gb. A gain of 24x, even if we would land at just a 10x gain of vram 240gb doesn't sound too bad. :D

But nvidia greed will probably keep the growth as slow as possible but even a 5x increase isn't too bad. Or we start seeing custom circuits that starts to force them to start going pop pop pop with the vram for us.

It's just the beginning :

What's wrong with moving? You will be closer to your waifu.

[-]

eggs-benedryl@reddit

I am also open to suggestions of how to avoid moving into the basement myself, so let me know :"D

Is this for fun or do you make money from a service you offer?

[-]

rustedrobot@reddit

> I am also open to suggestions of how to avoid moving into the basement myself, so let me know :"D

Show per posts of machines much more expensive than yours and show her it could have been much worse. XD

This makes my 12x look (slightly) tame.

What are you using to power everything? I've got 3x EVGA 1600w+ Gold PSUs for the 12 3090s and have found that any time I'm doing anything taxing I trip the protection circuitry in them. Running 3x 3090s per PSU seems to be working well so far.

Are you managing full PCIe4 speeds for all cards?

[-]

XMasterrrr@reddit (OP)

Show her posts of machines much more expensive than yours to demonstrate that it could have been much worse. XD

But babe, I am not as bad as the guy with 8x H100 stuck on his hand, she definitely wouldn't appreciate that 😂

On my 8x I went for 3x Superflower 1600w Platinum. Superflower are the manufacturer of Evga's PSUs and they're really good.

Now with the upgrade, I am going for 5x 1600w. And yes, managing full PCIe4 speeds for all cards, I plan on writing extensively on that in my upcoming blogpost this weekend.

[-]

rustedrobot@reddit

Sweet! Can't wait to read it. Def need to unblock a few bottlenecks in my rig.

As someone who's had trouble running 3 cards on PCI-E, I'd be interested to hear what you're doing there. I'm currently looking at using one of the extra NVME slots to run a PCI-E adapter.

[-]

L0WGMAN@reddit

This is great! I started playing with agent zero that the creator posted here and GitHub a while back, I love seeing similar constructions! And the hardware!

[-]

rustedrobot@reddit

Looks like as of BIOS 3.70 the ROMED8-T2 has rebar support:
https://www.asrockrack.com/general/productdetail.asp?Model=ROMED8-2T#Download

I went with the ROMED8-T2 over the H12SSL primarily because I wanted 12x GPUs and it has 7 PCIe4 16x slots that I could bifurcate. The H12SSL only has 5 16x slots and 2 8x slots. The seventh slot on my rig runs a 4x NVME card. I couldn't do all that on the H12SSL.

[-]

kryptkpr@reddit

I don't know how I missed the official rebar on this one, thanks so much!

These boards are an extra $200 but you do get the two full x16 vs the x8 on the Supermicro 🤔

Did you observe any difference with riser/redriver compatibility between the two boards? I got some cheap-ass dual width x8x8 boards on top of 15-20cm "pcie4" risers from AliExpress, not exactly premium gear over here

[-]

robogame_dev@reddit

I like the skeletal setup!

Let me rewrite that in a more positive way:

Just make sure you have a good answer when she asks "and what did you get for me?"

[-]

NobleKale@reddit

Just make sure you have a good answer when she asks "and what did you get for me?"

Like I have recordings of my professor's class, which I turned them into texts via WhisperX, but I am not able to feed any model. Even if I feed it, like in Gpt4all, I am not getting anything out any useful, like just summarise what he taught. Nothing useful. I tried LLama3.2 3B instruct, it does talk very good and unique but it is not working as I wanted. Maybe I did something wrong or maybe I should forget all these and make notes in class...😞🥲

[-]

LifeTitle3951@reddit

Have you tried notebooklm?

[-]

assert dominance

it's your money bro

it's your way or the highway

unless you live in a state like california, if then, gg

[-]

visarga@reddit

You scratched the table, u're in trouble.

[-]

Street-Coyote9075@reddit

On the bright side, you will not need to run the heat in your house this winter.

[-]

https://www.voxon.co/product-page/voxon-vx2

[-]

bosbrand@reddit

Visualize the words 'electric bill'...

[-]

tailcallrecursion@reddit

Now you need to explain her to this.

[-]

Curly_Grass_2296@reddit

Hey i am a total noob to the world of LocalLLaMA, may i ask what do you want to achieve by such an investment? I am asking out of pure curiosity since i see this and makes me wonder what kind of results i can expect from running localLlama on my gaming pc vs this powerhouse lol

[-]

XMasterrrr@reddit (OP)

I am currently working on a project that requires both batch inference and training. I did the math and I would have burned the same amount of money in a few months of compute renting so this was the right move for me.

shulke@reddit

The question she should ask is why not 4090 super

[-]

lanbanger@reddit

Because millionaire vs billionaire, I guess.

[-]

polikles@reddit

yes hun/mom, I need this for my school project

!or: yes, I need this to make AI to fight against syndicate aiming for creating an evil AI to take over the world!<

Lissanro@reddit

Nice! But I counted 14 cards, I suggest you to get 2 more for a nice power of two quantity (16). It would be perfect then.

Well they aren't building them anymore - neither 3090 or 4090, and the big offload from crypto coin boom is long past.

XMasterrrr@reddit (OP)

de4dee@reddit

RadSwag21@reddit

Can you really get more LLM power than Claude 3.5 or 4o out of these?

gamers then: grrr those pesky cryptominers!!

Me, trying to snag a 24GB GPU to run a super slow LLM locally but finding it too pricey:

Well.. she's an AI, right? So if you just start a new chat, she won't remember.

[-]

ares0027@reddit

My thought process;

fk thats too many
oh wait they are evga this faek
oooooh this is a coin miner using old pics
ah? I have the same pot set

(This process has nothing to do with. This is how my stupid “brain” worked and just wanted to share. Not accusing anyone anything or something)

[-]

Rndmdvlpr@reddit

Are we talking about your wife or the beast of an AI girlfriend you made?

[-]

Illustrious-Lake2603@reddit

Daang. I cant even get one @___@

[-]

ThePloppist@reddit

Even buying them at a steep discount this is going to be expensive.

I might be too drunk to do math right now, but that sounds like about twice the cost of current API pricing over a period of 5 years. Not terrible for controlling your own infrastructure and and guaranteed privacy, but still pretty rough.

On the other hand, that's roughly half the training data of llama3 in 5 years, literally made in your basement. It kind of puts things in perspective.

[-]

Pedalnomica@reddit

Hobby, and privacy are big ones, but the math can work out on the cost side if you are frequently inferencing, especially with large batches. Like, if you want to use an LLM to monitor something all day every day.

E.g. Qwen2-VL, count the squirrels you see on my security cameras -> LLama 405B, tell Rex he's a good boy and how many squirrels are outside -> TTS

The API prices are often pretty steep. However, maybe you can find free models on OpenRouter that do what you need.

[-]

Select-Career-2947@reddit

Probably they’re running a business that utilises them for R&D or customer data they needs to be kept private

[-]

EconomyPrior5809@reddit

yep, grinding through tens of thousands of legal documents, etc.

[-]

weallwinoneday@reddit

Whats going on here

[-]

Darkz0r@reddit

That's amazing.

Could you use that to build your dream game? Making the agents present you each major decision and how it's implemented in the game? Then you could approve or not and keep building.

Oh damn! You guys have figured out how to run multiple GPUs for models?! Damn I’ve been away a minute and things get better so fast!

kremlinhelpdesk@reddit

if I had infinite money, I'd do that, use all of them but one on a local server

"When I win the lottery, I won't say anything, but there will be signs."

falls asleep on a pile of 3090s

[-]