BoogerheadCult

Whatever you say, LLM model is use-case specific, I guess with your intelligence, you work on low level project that such a bad model is good enough for you. Bet you don't even know how to eval models properly to learn their limitations. 🤡

Hot garbage, tried and none of these models so far impressed me.

Microslop can't even get their shit AI on Windoze done right, these gonna be another disasters on the making.

Hell no, get yourself some used 3090 and build the rest is a lot cheaper. Not to mention it is a laptop so running it plugged in will cause very fast battery degradation and you will have a throttled useless paperweight after a few years once the battery is dead.

Nobody is using a Mac laptop to run LLM, it is bad, battery and thermal throttling. Mac Studio is more inline priced with the specs you mentioned.

Looks like how unemployment rate is computed, it was so easy to manipulate the numbers. Here in the US we have thousands of layoffs every other weeks, why would anybody with some common sense think the number is so little changed ? Looks at the Layoffs and the cscareerquestions subs, lots are being cooked. It is not so bad because the stock market is ATH due to AI hypes, if stock market crashes, lots more gonna feel the pain. If you want to buy this overpriced POS, then go for it. Many here don't see the appeals so they said no thanks. Sounds like you are just trying to convince yourself on a bad buy. For example 5k for 5090 that was 2k MSRP, sure I understand prices went up a bit but not 2.5x, that makes no sense for 32GB VRAM cards. You NoVidia shills just lost your god darn minds.

Everybody in the US knows the guy in the White House cooked up the stats. True unemployment rate is somewhere about 30%.

Right ? Better resale value too. All these Nvidia shills and people buying these overpriced garbage has lost their god darn minds.

Forgot the $3000 price tags. They think everybody gonna go rush out and buy this shit. LOL.

Need to do two things: 1) Use the last Nvidia driver that works with both cards 2) Use an older cuda-toolkit. 3) Compile llama cpp for multiple nvidia architecture. If it is a pain, dust off an old PC, put the older cards in the old PCs then use RPC comnection.

Dumb tests, because you are not pushing the system enough so there is not enough differences, on resource-constrained systems such as running very large models while trying to squeeze every last drop out of your RAM and VRAM, then Linux always shines. There is no coincidence that you can allocate more VRAM and run larger model on Ryzen Max in Linux but can't do it on Windows. Sorry OP, you are not technical enough for this kind of comparisons, just stay with Windows, it is for low IQ people anyway.

One is an old NUC I have laying around, I should upgrade that one soon, at least it has USB 3.0 so I bought a 2.5G USB ethernet and use that to link them together.

Helps with initial layer loading and I am obsessed with maxing out things that I can.

There's no such things. Claude already made this extremely easy to setup, if that's still a challenge then it's not for you, just give up on the idea.

Looks up using llama cpp over rpc. I got Claude to help me setting it up and it's pretty sweet. Hooking up 2x3090 with 2xR9700 across two machine over 2.5G ethernet. Gonna upgrade that soon to 10G.

https://preview.redd.it/qwg5k2jsm54h1.jpeg?width=1080&format=pjpg&auto=webp&s=bc7922bce027facc113d3e4315807216d2b6050f

So basically a dumbed-down Qwen to shill for Ponzi Buttcoin ? Count me in, I am so excited for this garbage /s

The people who can do it probably already worked at or poached by big AU firms making 500k to 7 figure. Why would they want to do all that for free ? Not to mention the hardware required, you need a huge hardware farm, only FAANGs can afford. Not even small to mid size companies can do that from scratch. So if just follow the money, you will get the answers.

Who cares, IBM is full of diversity hires, they are not going to deliver anything worth noting anyway.

bullshit, I got my R9700 configured and it outperformed all those except the 5090, skill issues. But seeing that you got more money than IQ then go ahead, be another NVIDIA sheep to the slaughter.

Neither of them, for the amount of VRAM you got, it is an extremely bad deal, blame it on the NVIDIA fanboys. Just have Claude help you with installing drivers and figure out how to get alternative cards working, way more cost effective.

Cheaper because your online tokens are currently VC-subsidized. Once you run your own model, you will see that it costs tons to just run anything decent. So technically it's still cheaper, hosting your own model is the true cost and the token cost you get with Gemini, Claude are fake cost. It doesn't reflect the true cost.

Nope, the NVIDIA fanboys are pushing prices to very unreasonable levels, $/VRAM is almost 2x, 3x competitors, I am looking at other alternatives now. The breaking point for me was seeing how badly used 3090 which sellers didn't even hesitate to disclose that they were used for mining now being sold for $1200. You gotta be kidding me.

Qwen has different set of defaults for different tasks, sorry but I think you are just an idiot who doesn't know what it is doing, hence rendering your little "experiment" useless.

It is the llama.cpp launch parameters, it has nothing to do with your script. I start to think either you are a Google shill or just an idiot who doesn't know what it is talking about.

If you spend 10k on hardware to run some of the most capable models out there right now such as the Qwen3.6 27B, sorry to say that you are an idiot. I spent no more than 2k and I can get it to run very smoothly and it already reduced my reliance on Claude, which allows me to reduce my monthly subscription to $20/month plan. Used 3090 go for around 1k, rest 1k can be on other hardware, which is doable.

Funny how you think your Claude subscription will stay the same. Right now most of your tokens are VC subsidized. Try running these yourself and you will realize the cost to run LLM model far exceeds what they are charging you for. When all the LLM providers increases their prices, there will be a huge surge in hardware demands and prices gonna shoot up. Good luck getting a rig at that point. Or keep paying $500 to keep your Claude subscription.

Biggest fad ever and so many stupid idiots installed this security risk on their personal device. Almost a kind of social experiments.