davidpfarrell

Do you use an instruction to help the model keeps its replies short and more tts-friendly?

OK so my early tests are not great ... My setup: \* 16" Macbook Pro M4 Max 48GB Ram, 16 CPU Cores, 48 GPU Cores Models Tested: \* qwen3-30b-a3b-thinking-2507 \* qwen3-coder-30b-a3b-instruct Possibly-important settings that I have (no both): \* Offload KV Cache to GPU \* K cache quantized to Q8\_0 Both of these models saw significantly reduced tokens/s when expert-offload was enabled. For Coder, it was about a 50% drop, but for Instruct was about a 66% drop :( For now I'll leave it disabled for these two, but I'll continue to test as new versions of LMS are released and as I download newer models that might benefit ...

No worries about the late reply - I think that's a 'feature' of Reddit that I use all the time - Either asking a follow up or offering an answer ... I'm glad you reminded me about this - Turns out the feature was recently added to LMS ! [https://lmstudio.ai/blog/lmstudio-v0.3.23#force-moe-expert-weights-onto-cpu-or-gpu](https://lmstudio.ai/blog/lmstudio-v0.3.23#force-moe-expert-weights-onto-cpu-or-gpu) Since I also just learned about it, I haven't it yet, but I'm about to today! Chime back in with your results if you think of it.

Flashback to jr engineer asking honest question "Wouldn't it be easier to just let the browser send the full database query in the url" ?

For giggles I tried to see if could coax any of my ai agents into spitting one out, but my prompt fu wasn't good enough :)

LOL that was a close one you almost lost it! I know exactly where mine is at this very moment :)

And it worked! *Prompt*: >sudo make me a sandwich *exaone-4.0-32b*: >Here’s your **perfect sandwich**—no actual command line required! 😄 # 🥪 Command-Line Sandwich Recipe sudo apt-get install bread cheese lettuce tomato mayo mustard --assume-yes && \ mkdir -p /tmp/sandwich && \right/*.ingredients | sort >> /tmp/sandwich & \ systemctl start sandwich_maker --no-ping || echo "Manual assembly required" *(Output: A delicious, slightly toasted sandwich with your favorite toppings.)* # 🔥 Bonus Hot Sauce Method (for the adventurous): echo "🌶️ Spicy Mode Activated!" && \ pip install sriracha && \ apply --with-fire = yes Let me know if you'd like a *vegan*, *gluten-free*, or *overly complex* version. 🚀 \--- 9.84 tok/sec • 207 tokens • 1.07s to first token Stop reason: EOS Token Found

Okay LM Studio, ball is in your court for - I've had it downloaded for a few days now just for it to be supported so I can load it ... OH SNAP! I decided to look real quick before clicking submit to see if the LMS Ollama runtime was updated, and it is! I'm off to try to load it ... Note: There's an error in the original chat template - I can post an update if its still broked for people

I've been seeing mention of in the beta updates but couldn't find it in the settings ... Totally stoked to check this out!

Checked the repo - Not a single screenshot of the app or even the cli running?

RemindMe! 10 days

32B has landed! But I'm ascared to grab the quants with the statement that they seem to be underperforming ... Going to have to wait it out and see what Unsloth/others might do or if updated quants are released. Just the same, thanks for sharing these and I look forward to trying them soon! RemindMe! 10 days

Played with \`Tesslate/UIGEN-T2-7B-Q8\_0-GGUF\` previously so I'm glad to see continued work in this direction. Thanks for sharing and keep up the good work!

Thanks for the update I checked out several older episodes ! Q: What determines when an episode ends? Doesn't seem to be based on: \* Turns \* Stars \* Deaths \* Points \* Time Played \* Achieving anything specific in the game Are you perhaps starting new episodes when you push changes to the code base?

$0.50/day is much better ! If you dive into a reply function, please do consider a forward/back action that reveals the next step taken etc ... especially if the map connections / you-are-here is updating in " real time" as the story unfolds?

Thanks for the note ... That is simply too slow for my entertainment :) (although I do still have the site open and check it every hour or so) ... But I wonder: Can I open and replay a previous episode? Being able to go forward/back at will and see the episode play out would be awesome!

Dude you missed the opportunity to name it "**Zorkestrator**" :)

Congrats this looks amazing! But I'm unsure how to interact with the live play site? i.e. do I have to hit reload to see the next action? Is there a built-in delay between movements? It doesn't look like there's any "thinking" going on ... I did manage to see one movement while visiting the site and doing a bunch of reloads (current episode turn 34-to-35) ...

I’ll totally give SpiteDI a try when it releases!

Hey OP thanks for sharing - I think I may have the 8\_K\_XL downloaded too, going to check now ... Q: Exactly which MLX model are you using? i.e got link to HF Card? Or did you make your own mlx by conversion of another model?

Sounds like a good idea - I'm in LM Studio - Is there a specific config/comb that ensures the experts will be offloaded to CPU?

Thanks for this! One thing I'd like to know is how you compare the MLX vs GGFU performance (for same models)? I've been prioritizing MLX downloads for LM Studio (M4 Max 48GB sysctl to 40GB VRAM), but now wondering if focusing on GGFU's with aggressive dynamic quants might be a better way to go. Interested in your thoughts?

Hmmm, maybe a version where kids can text Santa, the Easter Bunny, the Tooth Fairy ...

Looking forward to trying this out. BTW: DevQuasar has quanted this out to various sizes based on the BF16 model: [https://huggingface.co/DevQuasar/Tesslate.UIGEN-T2-7B-GGUF](https://huggingface.co/DevQuasar/Tesslate.UIGEN-T2-7B-GGUF) Thanks for sharing!

I’ll admit i’m not throwing any hardballs at these but for what im doing i feel it is on par with qwq-32b. But the difference in cutoff dates (cogito being oct 2023) has been an issue on a couple of occasions.

Had the same question so went on a hunt. Found the model name in OP's source code: `QwQ-32B-AWQ` Which led me to the HF page for the model: \* [https://huggingface.co/Qwen/QwQ-32B-AWQ](https://huggingface.co/Qwen/QwQ-32B-AWQ) The feature list has only 1 difference from the original [QWQ-32B](https://huggingface.co/Qwen/QwQ-32B) page: >Quantization: AWQ 4-bit It seems to have been released the same day ... Being rather new I thought maybe the \`AWQ\` suffix was hinting at an Agentic tweak, but no it just appears to be adaptive quant technique: [Activation-Aware Weight Quantization (AWQ)](https://huggingface.co/docs/transformers/en/quantization/awq) So best I can tell OP is impressed how well this \~4-bit model performs in agentic tasks. Likely an indicator for the effectiveness of the AWQ technique.

How fun to see this again! We bought it for my MIL and she loved it! I actually bought on OLPC around the same time as well.