open source model small enough to run on a single 3090 performing WAY better in most benchmarks than the ultra proprietary closed source state of the art model from only a couple months ago

Posted by pigeon57434@reddit | LocalLLaMA | View on Reddit | 22 comments

https://preview.redd.it/tcevum2it6ee1.png?width=2048&format=png&auto=webp&s=efffd7c575dc52e3a3c516e2b4e92c09e67c6c09

Reply to Post

22 Comments

[-]

Fantastic_Affect_485@reddit

Do you seriously think you can beat claude with a single rtx3090? Have you used claude even a single time? Is 24GB of VRAM enough to beat such a model? I do NOT care what these benchmarks are trying to push, it is currently impossible to beat claude 3.5 (even the old version) with 24 GB of VRAM. (And before you ask, yes I have used all the DeepSeek models that run on 24GB VRAM, none of them come close to beating claude in anything, period).

[-]

MixtureOfAmateurs@reddit

Do you seriously think you can beat llama 1 70b with 8Gb of VRAM? Have you used llama 70b a single time? Progress progresses bro. 3.5 sonnet is getting old, I don't think it's been practically overtaken yet by this distillation, but your comment is gonna sound like a shit post in 6 months

[-]

Oatilis@reddit

I think the architecture and training data are arguably more important than model size to an extent. We can see new models outperforming larger models all the time. There will definitely be a continued effort to compress models while retaining or even improving abilities.

[-]

MixtureOfAmateurs@reddit

!remindme 6 months

[-]

RemindMeBot@reddit

I will be messaging you in 6 months on [**2025-07-21 12:13:29 UTC**](http://www.wolframalpha.com/input/?i=2025-07-21%2012:13:29%20UTC%20To%20Local%20Time) to remind you of [**this link**](https://www.reddit.com/r/LocalLLaMA/comments/1i5wam1/open_source_model_small_enough_to_run_on_a_single/m8c4318/?context=3) [**CLICK THIS LINK**](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5Bhttps%3A%2F%2Fwww.reddit.com%2Fr%2FLocalLLaMA%2Fcomments%2F1i5wam1%2Fopen_source_model_small_enough_to_run_on_a_single%2Fm8c4318%2F%5D%0A%0ARemindMe%21%202025-07-21%2012%3A13%3A29%20UTC) to send a PM to also be reminded and to reduce spam. ^(Parent commenter can ) [^(delete this message to hide from others.)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Delete%20Comment&message=Delete%21%201i5wam1) ***** |[^(Info)](https://www.reddit.com/r/RemindMeBot/comments/e1bko7/remindmebot_info_v21/)|[^(Custom)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5BLink%20or%20message%20inside%20square%20brackets%5D%0A%0ARemindMe%21%20Time%20period%20here)|[^(Your Reminders)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=List%20Of%20Reminders&message=MyReminders%21)|[^(Feedback)](https://www.reddit.com/message/compose/?to=Watchful1&subject=RemindMeBot%20Feedback)| |-|-|-|-|

[-]

raiffuvar@reddit

No, it won't.

[-]

NegativeCrew6125@reddit

let's wait 6 months then

[-]

Previous_Street6189@reddit

He says currently

[-]

pigeon57434@reddit (OP)

Claude is really not as good as everyone insists all day long. And yes, I used to be a paying subscriber, and it was fairly recently, so I was using Sonnet 3.6. This is not to say R1-32B is better, but I really just don’t get why everyone praises Anthropic so damn much.

[-]

ThePixelHunter@reddit

Holy fuck that coding score changes everything

[-]

Any_Pressure4251@reddit

Until you try the models and they fall short.

[-]

PeachScary413@reddit

But they did so well on the benchmarks that they totally didn't include in their test sets 🥲

[-]

Specter_Origin@reddit

Context window is bit problem, you can barely chat with 2 responses, specially cause reasoning is also fed back into the context. With complex problems it is only able to do 1 response.

[-]

Enough-Meringue4745@reddit

Yeah, we’ll need a reasoning summary context system

[-]

Kep0a@reddit

I liked a novel idea that someone mentioned here awhile ago; 'reasoning' developing into it's own token efficient language.

[-]

10minOfNamingMyAcc@reddit

This, have it think internally (but with a brainchip to read its mind) instead of externally, kinda like humans do.

[-]

fogandafterimages@reddit

Just you wait until the RWKV guys get their hands on it. They take strong Qwen models, port over the MLP as-is, and fine-tune RWKV's recurrence mechanism to approximate the attention used in Qwen, and get *rather* nice results. Aaaand recurrent models *don't have a context window*. They start to forget stuff after a few thousand tokens, but they don't get slower or go out-or-distribution and bork like transformers do.

[-]

open source model small enough to run on a single 3090 performing WAY better in most benchmarks than the ultra proprietary closed source state of the art model from only a couple months ago

Reply to Post

22 Comments

Fantastic_Affect_485@reddit

MixtureOfAmateurs@reddit

Oatilis@reddit

MixtureOfAmateurs@reddit

RemindMeBot@reddit

raiffuvar@reddit

NegativeCrew6125@reddit

Previous_Street6189@reddit

pigeon57434@reddit (OP)

ThePixelHunter@reddit

Any_Pressure4251@reddit

PeachScary413@reddit

Specter_Origin@reddit

Enough-Meringue4745@reddit

Kep0a@reddit

10minOfNamingMyAcc@reddit

fogandafterimages@reddit

SEND_ME_YOUR_POTATOS@reddit

Dead_Internet_Theory@reddit

ThePixelHunter@reddit

LocoLanguageModel@reddit

mrjackspade@reddit