[-]

Conscious_Cut_6144@reddit

This guy has a Llama4 style architecture with decnetly large shared expert (slightly over 1/2 of the 14B is shared)
Should run well on gaming rigs with 128GB ram.

[-]

CheatCodesOfLife@reddit

It's a cut-down DeepSeek-V3 architecture with Qwen2 tokenizer.

[-]

Particular_Rip1032@reddit

1 year old cutoff. Still pretty new in LLM standards.

[-]

CheatCodesOfLife@reddit

That's great news for creative writing. Less SLOP in the "no synthetic data in pretraining" dataset.

[-]

datbackup@reddit

14B active 142B total moe

Their MMLU benchmark says it edges out Qwen3 235B…

I chatted with it on the hf space for a sec, I am optimistic on this one and looking forward to llama.cpp support / mlx conversions

[-]

SkyFeistyLlama8@reddit

142B total? 72 GB RAM needed at q4 smh fml roflmao

I guess you could lobotomize it to q2.

[-]

Thomas-Lore@reddit

With only 14B active it will work on CPU only at decent speeds on DDR5.

[-]

colin_colout@reddit

This. I have a low power mini PC (8845hs with 96gb ram) and can't wait to get this going.

Prompt processing will still suck, but on that thing it always does (thank the maker for kv cache)

[-]

Pardon the dumb question, haven't dabbled with MoE that much, but the whole Model still needs to be loaded in RAM, right, even when only 12B are active? So with 64GB Ram (+8 Vram) I'm still without luck, correct?

[-]

i-eat-kittens@reddit

Only the active nodes need to be loaded afaik. There were people who ran llama4 mostly from disk, so if you have fast enough drives and IO it could be "usable".

My desktop is also 8+64, and I'll be giving it a try just for the lols. If I understand the top comment correctly, I'll try putting the 6 routing layers and two shared experts on my gpu and run the rest from ram/ssd.

I do wish for recent 7A/70B model or something, with routing layers and shared experts that will fit on my gpu. Putting 8GB worth of 30B-A3B into vram doesn't seem too useful.

[-]

SkyFeistyLlama8@reddit

I've tried running Llama Scout from SSD. It's unusable like that, fun to show it could be done but I would never actually use it that way. On my 64 GB laptop, I ended up using Scout Q2 to get it to fit completely in RAM.

MOEs still need to be completely loaded into fast RAM (VRAM or unified high-speed RAM for iGPUs) if you want decent performance. Otherwise, loading from SSD is at least 10x slower. An MOE with 40-50B total would be perfect for smaller systems because it would take 20-25GB RAM at q4 and it would be super fast with only 6B active parameters.

[-]

colin_colout@reddit

Not exactly but it helps. I could run 1 bit quantized llama maverick at a few tk/s, and I don't have quite enough RAM for that.

Llama.cpp is quite good at keeping the most important experts in memory. Clearly it is much better to keep everything in fast memory, but for the models I tried it's not so bad (given the situation of course).

Try it.

[-]

Calcidiol@reddit

You'll have (64+8) RAM/VRAM - overhead for OS and context etc. (-10) so 62 GBy free or so maybe so under 3.5 bits / weight could work without overloading RAM beyond this level, so look at maybe a Q3 XXS GGUF model version or something like that and see if that's good enough quality.

[-]

SkyFeistyLlama8@reddit

I guess the downvoters failed reading comprehension.

You still have to load the entire model into some kind of RAM, whether that's HBM VRAM or unified RAM on Apple Silicon or Snapdragon X or Strix Halo. Unless you want potato speed running the model from disk and having to load layers from disk into RAM on every forward pass.

Once it's in RAM, whatever kind of RAM you have, then you can use a GPU or CPU or NPU to process the model.

[-]

ROOFisonFIRE_usa@reddit

sorry 32gb is not the sweet spot unfortunately. 48-96gb is more appropriate. 32gb is just a teaser.

[-]

SkyFeistyLlama8@reddit

I'm thinking more about laptop inference like on these new CoPilot PCs. 16 GB RAM is the default config on those and 32 GB is an expensive upgrade. 96 GB isn't even available on most laptop chipsets like on Intel Lunar Lake or Snapdragon X.

[-]

ROOFisonFIRE_usa@reddit

We're still a couple years away from solid local model performance on laptops. My take on that is it's better to pick up a thunderbolt egpu enclosure than run any kind of meaningful GPU in a laptop form factor. Just asking for trouble and an expensive repair with that much heat and power draw on a laptop.

[-]

Thomas-Lore@reddit

I knew I will regret getting 2x32 DDR5 instead of 2*48.

[-]

shing3232@reddit

It's a baby between qwen3 and deepseek

[-]

Calcidiol@reddit

Scout's big brother. Or maybe that's backwards...

[-]

shing3232@reddit

They reuse parts from qwen and deepseek which is funny

[-]

silenceimpaired@reddit

Where did you see that?

[-]

Entubulated@reddit

They re-use architectural features from multiple models, which has advantages including reducing effort their initial design phase before getting to model training and that tools like llama.cpp and downstream should be able to add support quickly. They also briefly discuss plans on architectural changes somewhere near the end of the whitepaper. Mostly adding in support for more attention mechanisms.
https://github.com/rednote-hilab/dots.llm1/blob/main/dots1_tech_report.pdf

[-]

silenceimpaired@reddit

Thanks for sharing.

[-]

SelectBody2853@reddit

[-]

SelectBody2853@reddit

[-]

SelectBody2853@reddit

[-]

stefan_evm@reddit

Unfortunately, no multilingual support (only Englisch and Chinese). Tests and can confirm this.

[-]

Particular_Rip1032@reddit

>Englisch

Wie geht es dir?

[-]

Particular_Rip1032@reddit

>Englisch

Wie geht es dir?

[-]

FlamaVadim@reddit

In my european language it is quite ok, but not ideal.

[-]

stefan_evm@reddit

Yes, that's what I mean. Other models (e.g. Gemma, Llama, Mixtral) much much better language capabilities.

[-]

captain_shane@reddit

How long does china keep releasing top end models? All the way until agi?

[-]

Ok_Cow1976@reddit

seems very likely so far

[-]

captain_shane@reddit

Then they either save the world or we're looking at ww3. Zero chance the west allows china to release extremely powerful models into the wild. They'll do death penalties or something if you're caught with a blacklisted model.

[-]

Ok_Cow1976@reddit

what's the rationale for ww3? I don't get it. people seems to be happy about excellent ai assistants

[-]

captain_shane@reddit

Cyber warfare. Agi would let people hack their own banks. That's just taste of what it could do. Imagine sending out a million agents that are all running sophisticated and personalized ramsomware scams? That's just a tip of the iceberg.

[-]

eli_pizza@reddit

AGI isn’t real

[-]

UnionCounty22@reddit

A blimp and hot air balloon is the ONLY way we’ll ever fly! Mark my words!!

[-]

captain_shane@reddit

Depends on the definition, but I'd say we're already there. It's smarter than every human on the planet in a general way knowledge wise, not logic wise. I would say we get to asi when it's logically smarter and knowledgably smarter than every person on the planet. Consciousness is probably impossible to create from machinery, but I'm not sure if that's what you're talking about.

[-]

Soft-Ad4690@reddit

Wikipedia is "smarter" than every human on planet earth knowledge wise.

[-]

Ok_Cow1976@reddit

I don't find this could be a problem. I mean, we are already overwhelmed by junk mails.

[-]

captain_shane@reddit

I also don't really want to get into the political side of this and risk banning. But video generated propaganda is also a reason to start ww3. The pen is mightier than the sword and if people can use ai to power up those words and use ai actors to spread the message, that's a powerful tool for the masses.

[-]

Ok_Cow1976@reddit

I have to admit you have better imagination, fantasy than ai.

[-]

captain_shane@reddit

Are people domesticated animals at this point? Maybe. But if they aren't then ai actors spreading messages could easily rile people up to do anything. Who the fuck are the influencers today? Hasan Pike? Charlie Cuck? Tucker Carlson? AOC? You're telling me that carefully crafted propaganda tailored specifically for a target audience with a specifically curated ai actor couldn't rile those people into action?

We already see what the news tells people and quite frankly all those newscasters could be ai and it wouldn't make any difference at all. Someone listening to sean hannity or rachel maddow is no different than listening to joe blow ai, except joe blow is far more sophisticated, handsome and persuasive.

[-]

toothpastespiders@reddit

I think the issue is that we're already mentally owned and branded. I'm skeptical that any narrative could shake most of us out of our established, unconscious loyalties.

[-]

Ok_Cow1976@reddit

I don't find this a problem either. people will adapt to new situation quickly. Wars are typically generated by economic problems and people are utilizes by politicians. So I would pay more attention to economic consequences of AGI.

[-]

captain_shane@reddit

Wars are usually fought over resources. What do you think these provocations around the world are? China's grabbing resources in africa and se asia, the US is broadcasting that is wants to do it in greenland and south america, russia is currently doing it in europe. Israel sure isn't fighting and genociding gazans over economics.

[-]

Ok_Cow1976@reddit

I would say some capitalists are too greedy.

[-]

captain_shane@reddit

Probably right, but that's what wars are fought over. Someone wanting someone else's stuff.

[-]

NegativeCrew6125@reddit

kind of irrelevant IMO, just scroll though your social media site of choice and you will find millions of people falling for nonsense, without AI even being involved

[-]

captain_shane@reddit

yeah, but you aren't inundated with junk mail that looks like what ai can generate. I think we're talking about different things anyway, I suppose I'm talking about ASI than when people discuss AGI.

[-]

Ok_Cow1976@reddit

what's ASI? But anyway, about AGI being open sourced, if it is harmful for western society it's also harmful for China as well. so it is more likely that the capitalists will unite to prevent it being open sourced. but ww3? I guess you just don't want to see Chinese companies excel. but why you? are you one of the capitalists?

[-]

captain_shane@reddit

I want to see china succeed. Them releasing powerful models into the wild is good for humanity, I just don't know how long that'll last.

[-]

Ok_Cow1976@reddit

ok, we don't have to spoil the vibe of being technical purely. I suggest we stop here.

[-]

captain_shane@reddit

Dude I'm literally anti-west at this point and left my country of origin, but china releasing agi/asi whatever you wanna call it into the wild wouldn't be a whole lot different than giving everyone a nuclear reactor. I support it, but I don't think the west will.

[-]

SidneyFong@reddit

I guess once DeepSeek started doing it, everyone else had to follow...

内卷 FTW

[-]

CheatCodesOfLife@reddit

It didn't know what "gguf when?" means when I prompted that out of the blue.

[-]

datbackup@reddit

I will take that as a positive — means it hasn’t been trained on reddit data

[-]

CheatCodesOfLife@reddit

Same here, I meant that as a positive as well (but I can see why it looks like I was criticizing it).

Fresh dataset! I'm really excited for finetuning support and gguf/awq/exl2 quants.

[-]

myvirtualrealitymask@reddit

[-]

cromagnone@reddit

I feel seen.

[-]

ortegaalfredo@reddit

I like it already.

[-]

silenceimpaired@reddit

Does this poem exist online because I’m shocked at how human like it is

[-]

myvirtualrealitymask@reddit

I have no idea, This is higher quality and better creatively than most models on the same prompt though, maybe because of high quality pre training data and/or lack of synthetic data use?

[-]

IrisColt@reddit

Obviosuly aligned.

[-]

silenceimpaired@reddit

Why do you feel that way?

[-]

gpupoor@reddit

LMAO I'm sold

[-]

cms2307@reddit

Looks SOTA to me

[-]

silenceimpaired@reddit

I asked for unique fantasy story concepts, and it gave me ones that do not sound anything like the previous typical ones I’ve gotten from larger models. How unique would you consider these?

Here are three unique fantasy novel ideas with distinct settings, themes, and twists:

1. "The Clockwork Kingdom"

Genre: Gaslamp Fantasy / Steampunk
Logline: In a world where time is a finite resource, people trade and steal it like currency—but those who hoard too much are hunted by the enigmatic Chrono Reapers, sentient timepieces that enforce balance. When a street thief accidentally steals a royal heirloom that can rewind time itself, she becomes the target of assassins, revolutionaries, and a vengeful prince who blames her family for his father’s death. But the true danger lies in what happens when time begins to unravel…

Themes: Time as currency, class disparity, sacrifice for power.
Twist: The Chrono Receptors are not enforcing balance—they’re consuming time to feed an ancient, waking god.

2. "The Bone Harp"

Genre: Dark Fairy Tale / Gothic Fantasy
Logline: A mute girl with a cursed voice is sold to a reclusive nobleman who collects relics of the dead. His latest acquisition, the Bone Harp, is said to summon the spirits of those who play it—but it also whispers secrets from beyond the grave. As she deciphers the harp’s cryptic songs, she uncovers a conspiracy involving a forgotten rebellion, her own buried past, and a love that defies death itself. But every note played risks tearing open the veil between worlds.

Themes: Silence as power, grief, forbidden love.
Twist: The nobleman isn’t human—he’s a bone collector, a being who harvests souls to keep the world from collapsing into chaos.

3. "The Witch of the Hollow Stars"

Genre: Space Fantasy / Cosmic Horror
Logline: On a dying colony planet, a disgraced astronomer discovers that the stars aren’t just distant suns—they’re the eyes of an eldritch entity watching from the void. When she’s accused of witchcraft after predicting a meteor strike, she’s saved by a nomadic cult that worships the stars as gods. But as she learns their secrets, she realizes their "gods" are parasites feeding on human suffering—and she might be the only one who can starve them out.

Themes: Faith vs. science, cosmic indifference, survival at any cost.
Twist: The "meteor strike" is the entity’s offspring, sent to terraform the planet for its own use.

Each of these has room for deep world-building, morally gray characters, and unique magic/technology systems. Would you like any of them expanded further?

[-]

nuclearbananana@reddit

Style is a lot like deepseek v3

[-]

silenceimpaired@reddit

I’ve heard good things about DeepSeek and fiction.

[-]

nuclearbananana@reddit

I've used it extensively. Great prose, pretty good ideas, good creativity, but dumb as rocks. All that benchmark maxing goes out the window when it gets into story writing mode.

[-]

silenceimpaired@reddit

I assume you mean Deepseek for “it”… anything better in your experience?

[-]

nuclearbananana@reddit

Sonnet is the gold standard (3.5 is still my favorite), but it's expensive. Deepseek is still the best for its price.

[-]

Lazy-Pattern-5171@reddit

I believe the first one already was a movie long ago. The last one does feel very unique but I’m not very well read in fantasy fiction yet.

[-]

MidAirRunner@reddit

Really? What movie? I might give it a watch lol.

[-]

Pedalnomica@reddit

In Time? (which totally should have been called Just In Time as it stars Justin Timberlake)

[-]

silenceimpaired@reddit

Ohhhhhhhhh. Lost Title opportunity

[-]

SteveRD1@reddit

Yeah when I read that 'unique' idea I recognized 80% of it right away.

[-]

teachersecret@reddit

Different, but, the other issue is that when you try again, you get similar things. For example, I asked for some and here's what I got: 1. The Clockwork Chimera

Setting: Luminara, a city where time flows backward in the wealthy districts, forward in the slums, and stagnates in the neutral "Golden Belt." The city is a patchwork of temporal anomalies, governed by the Temporal Accord, which forbids altering the flow of time outside one’s district.

Themes: Identity, free will vs. fate, and the cost of immortality.

Twist: The protagonist, a thief who steals memories instead of objects, discovers that the city’s ruling elite are artificial constructs—clockwork beings created by a forgotten alchemist to "preserve perfection." The true source of Luminara’s temporal chaos? A dying god trapped within the city’s core, feeding on stolen time.

Why It Stands Out: A blend of steampunk aesthetics and time-bending philosophy, with moral dilemmas about whether it’s better to live fast, slow, or not at all.

2. The Bone Witch

Setting: The Ashlands, a desert where bones sing and the dead do not rest. Theocracy reigns here, with the Singing Priests controlling the land using the "Echo Song," a magic that binds the spirits of the departed to serve the living.

Themes: Grief, rebellion against religious dogma, and the ethics of necromancy.

Twist: The protagonist, a young woman branded a heretic for refusing to bind her brother’s spirit, discovers she’s not resurrecting the dead—she’s calling back fragments of a long-dead world-eating entity. The Ashlands are a graveyard for its bones, and the priests have been unwittingly feeding its rebirth.

Why It Stands Out: A dark, lyrical take on necromancy with a Lovecraftian twist—raising the dead is less about reanimation and more about waking something that shouldn’t exist.

3. The Glass Witch

Setting: Veridia, a floating archipelago where islands are giant, hollowed-out crystals that refract emotions into visible light. The populace "harvests" these emotional hues for power, but overuse causes the islands to destabilize and fall from the sky.

Themes: Emotional labor, societal pressure, and the dangers of suppressing feelings.

Twist: The protagonist, a lowborn "Glass Witch" who can manipulate emotions but not feel them herself, uncovers that the ruling class’s "pure white" magic is a lie—they’ve been draining the emotions of an enslaved race hidden beneath the islands. The crystals aren’t natural—they’re the crystallized souls of the oppressed.

Why It Stands Out: A visually stunning world where magic is tied to psychology, with a brutal commentary on emotional exploitation.

Each of these offers a fresh twist on classic fantasy tropes while exploring deep thematic questions. Which one intrigues you the most? I can expand on any of them!

See how they have some similarities? :)

[-]

silenceimpaired@reddit

Sigh. I should have tried a few more times

[-]

WitAndWonder@reddit

I'd have to see the prompt, but these are excellent if the prompt was basic. I've prompted similar from other models but with fairly elaborate guiding to get rid of a lot of AIisms / Slop / Cliches.

[-]

silenceimpaired@reddit

All I said was “give me three unique fantasy novel ideas.”

[-]

XForceForbidden@reddit

This model gave an incorrect answer to an Nginx configuration question I specifically prepared.

The question wasn't clearly explained in the Nginx documentation—in fact, the documentation was somewhat misleading. However, by examining the source code, one could arrive at a clear answer.

For this question, qwen3-235b-a22b provided the correct answer without requiring reasoning mode, while qwen3-32b needed reasoning mode to answer correctly. (Interestingly, when I conducted tests in late February, only Claude 3.7 could answer correctly without reasoning mode; Grok, DeepSeek, and OpenAI all required reasoning mode. By late April, DeepSeek v3 0324 was also able to answer correctly.)

[-]

Former-Ad-5757@reddit

Read up on what a base model is. You are comparing apples with oranges

[-]

nuclearbananana@reddit

Not OP but it has been chat tuned though

[-]

Sicarius_The_First@reddit

Chinese guys keep on winning. Looks very interesting!

[-]

westsunset@reddit

IMO China's work in AI (and electric vehicles) has been the best PR they could hope for.

[-]

emprahsFury@reddit

It is good pr, but it's the result of a multi-trillion dollar economy being focused from the top-down with massive assists from economic espionage. The fact that singleton companies like OpenAI and Mistral and Google and Meta can all independently compete against that is better PR imo.

[-]

westsunset@reddit

Yes there is definitely a national push. That's not to say American doesn't have some incentive to help our companies, look at Trump's Beautiful Bill, but its on another level for China My point is a few years ago a lot of conversation was incapable of this kind of innovation. I do think the government involvement in China ultimately is a hindrance to their potential though.

[-]

foldl-li@reddit

too large to run locally. bye.

[-]

valdev@reddit

For you

[-]

Sudden-Lingonberry-8@reddit

Would you die if I took off that mask?

[-]

balianone@reddit

Tested it out, and it underperformed. Turns out it can't speak Balinese

[-]

Sudden-Lingonberry-8@reddit

time to put a lid on that model

[-]

silenceimpaired@reddit

I wish comparisons existed for quantizations of the models … like a Q4 of 72b vs Q3 of Rednote

[-]

fdg_avid@reddit

Much better world knowledge than Gemma 3 27B and Qwen 3 32B based on my limited testing.

[-]

TheRealMasonMac@reddit

Arguably better world knowledge than Qwen 3 200B.

[-]

Starcast@reddit

At 142B params I would hope so

[-]

starfries@reddit

Oh that's interesting. Will try it myself

[-]

georgejrjrjr@reddit

Notably, they are releasing a true base model (with no synthetic data), under a real open source license (which hasn't really happened since Nemotron-340B), *with intermediate checkpoints* --meaning it can be customized for just about any data distribution by annealing the learning rate on .

Underrated release, imo.

[-]

TheRealMasonMac@reddit

These model releases are worth gooning to.

[-]

bash99Ben@reddit

So maybe deepseek should realease a Deepseek-R1-Distilled-dots.llm1 ?

[-]

FullOf_Bad_Ideas@reddit

Yeah this is missing in Qwen and it will be a big deal.

[-]

starfries@reddit

Oh that's very cool actually. Guess we'll be seeing a lot of dots finetunes in the future.

[-]

Chromix_@reddit

They tried hard to find a benchmark for making their model appear as the best.

They compare their model MoE 142B-14A against Qwen3 235B-A22B base, not the (no)thinking version, which scores about 4 percent points higher in MMLU-Pro than the base version - which would break their nice looking graph. Also, Qwen3 14B which scores nicely in thinking mode is suspiciously absent - it'd probably get too close to their entry.

[-]

MKU64@reddit

They weren’t obviously going to compare their non-reasoning model to a reasoning model, like if R1 was there.

It’s not really either way about being better than Qwen3-235B alone, it’s a cheaper and smaller LLM for non-reasoning, we didn’t had one of ≈100B in a while and this one will do wonders for that.

[-]

Chromix_@reddit

Yes, apples to apples comparisons make sense, especially to fresh apples. Still it's useful for the big picture to see where it fits the fruit salad.

[-]

ortegaalfredo@reddit

I didn't knew qwen2.5-72B was so good, almost at qwe3-235B level.

[-]

Dr_Me_123@reddit

235B took the place of the original 72b. 72b was once even better than their commercial, closed-source, bigger model qwen-max at that time.

[-]

FullOf_Bad_Ideas@reddit

It is good at tasks where reasoning doesn't help (the Instruct version). As a base pre-trained model, it's very strong.

There are reasoning finetunes like YiXin 72B and they're very good IMO, though the inference of non-MoE reasoning models this size is slow, which is why I think this size is getting a bit less focus lately.

[-]

Chromix_@reddit

That depends on how you benchmark and where you look. If you look at the Qwen3 blog post, you can see that their 30B-A3B already beats 2.5-72B by a wide margin in multiple benchmarks.

[-]

starfries@reddit

Yeah wish I could see this plot with more Qwen3 models.

[-]

Final-Rush759@reddit

Based on the paper, it's very similar to Qwen3 32B in benchmark performances.

[-]

abskvrm@reddit

People would be raving had Llama been half as good as this one.

[-]

IrisColt@reddit

sigh...

[-]

pip25hu@reddit

Interesting, though inference by major frameworks being only in PR status at best will serve as a barrier to adoption.

[-]

Entubulated@reddit

That should change fairly quickly. Comments in the feature request for llama.cpp talk about how this model's architecture basically pastes together features from other model's architectures and that implementation should be rather straightforward. The transformers PR seems to be waiting for more test cases.

[-]

FrostyContribution35@reddit

Does this model have GQA or MLA? The paper said a "vanilla multi-head attention mechanism" with RMSNorm. How are they gonna keep the KV cache from growing exponentially with long prompts?

[-]

LoveThatCardboard@reddit

If the stats are true this is a big improvement on Qwen3 for Macbook enjoyers.

On a 128 GB MBP I have to run Qwen3 at 3-bit quantization and have a limited context. This should be able to have a decent context even at 4-bit.

[-]

AllanSundry2020@reddit

is there an mlx release of this?

[-]

colin_colout@reddit

What kind of prompt processing speeds do you get?

[-]

LoveThatCardboard@reddit

Not sure how to measure the prompt specifically but llama-bench reports 35 tokens/s in its first test and then segfaults.

[-]

FullOf_Bad_Ideas@reddit

It's really cool that they open source intermediate weights and base model.

Qwen3 32B base and Qwen3 235B base models don't exist in public - don't forget about this. Looking at benchmarks, their Instruct finetune is about equal to Qwen3 32B no-thinking - it may be actually disappointing to some that it's not better with 142B of weights. MoE hurts.

I can't wait for someone to add thinking to it.

[-]