<Think> toggle button for llama.cp web chat for QWEN3.6

Posted by ea_man@reddit | LocalLLaMA | View on Reddit | 21 comments

https://preview.redd.it/od6suf6j7g4h1.png?width=619&format=png&auto=webp&s=d31fb903ea68f58e3a641bfd275d59eeb5cce445 Missing a button in llama-serve webchat to toggle reasoning on/off like in LM Studio? This is a snippet that runs in [https://www.tampermonkey.net/](https://www.tampermonkey.net/) a browser extension that injects extra functionalities in existing web pages, so you can compile llama.cp every day without bothering patching, it stays in your browser. You need to install the extension and add this script: // ==UserScript== // llama-ui Reasoning Toggle // http://tampermonkey.net/ // 3.1 // Places the reasoning toggle inline with native button color mapping // Eaman // http://localhost:8080/* // http://127.0.0.1:8080/* // none // document-start // ==/UserScript== (function() { 'use strict'; window.__reasoningEnabled = (localStorage.getItem('qwen_reasoning') !== 'false'); // ========================================== // 1. NETWORK INTERCEPT // ========================================== const originalFetch = window.fetch; window.fetch = async function(...args) { const url = args[0]; const options = args[1]; if (typeof url === 'string' && (url.includes('/v1/chat/completions') || url.includes('/chat/completions')) && options && options.body) { try { let data = JSON.parse(options.body); if (!window.__reasoningEnabled) { if (!data.chat_template_kwargs) data.chat_template_kwargs = {}; data.chat_template_kwargs.enable_thinking = false; data.reasoning_budget = 0; } else { if (!data.chat_template_kwargs) data.chat_template_kwargs = {}; data.chat_template_kwargs.enable_thinking = true; } options.body = JSON.stringify(data); } catch (e) { console.error(e); } } return originalFetch.apply(this, args); }; // ========================================== // 2. NATIVE INLINE INJECTION // ========================================== function drawToggleBtn() { if (document.getElementById('llama-native-inline-toggle')) return; const plusBtn = document.querySelector('.file-upload-button'); if (!plusBtn || !plusBtn.parentNode) return; const btn = document.createElement('button'); btn.id = "llama-native-inline-toggle"; btn.type = "button"; updateButtonUI(btn, window.__reasoningEnabled); // Layout properties btn.style.display = 'inline-flex'; btn.style.alignItems = 'center'; btn.style.justifyContent = 'center'; btn.style.shrink = '0'; // Circular pill framing matching native elements btn.style.padding = '0px 10px'; btn.style.fontSize = '10px'; btn.style.borderRadius = '9999px'; btn.style.cursor = 'pointer'; btn.style.fontWeight = 'bold'; btn.style.height = '32px'; btn.style.whiteSpace = 'nowrap'; btn.style.transition = 'all 0.15s ease'; btn.addEventListener('mousedown', (e) => { e.preventDefault(); }); btn.addEventListener('click', (e) => { e.preventDefault(); e.stopPropagation(); window.__reasoningEnabled = !window.__reasoningEnabled; localStorage.setItem('qwen_reasoning', window.__reasoningEnabled); updateButtonUI(btn, window.__reasoningEnabled); const textarea = document.querySelector('textarea'); if (textarea) textarea.focus(); }); plusBtn.parentNode.insertBefore(btn, plusBtn.nextSibling); } function updateButtonUI(element, enabled) { if (enabled) { // ON State: Pure White background, dark text (Matches Submit Arrow) element.innerText = "🧠 ON"; element.title = "Reasoning Enabled"; element.style.backgroundColor = '#ffffff'; element.style.color = '#121212'; element.style.border = '1px solid #ffffff'; } else { // OFF State: Muted Dark Gray background, white text (Matches Plus Button) element.innerText = "⚡ OFF"; element.title = "Reasoning Disabled"; element.style.backgroundColor = 'rgba(255, 255, 255, 0.15)'; element.style.color = '#ffffff'; element.style.border = '1px solid rgba(255, 255, 255, 0.05)'; } } const observer = new MutationObserver(() => { drawToggleBtn(); }); observer.observe(document.body, { childList: true, subtree: true }); })(); *What does it do?* Button press changes the state of llama.cp `chat_template_kwargs` which is like passing the old deprecated --chat-template-kwargs '{"enable_thinking":false}' on launch or setting you own setting -> custom json to: { "chat_template_kwargs": { "enable_thinking": false }, "reasoning_budget": 0 } Disclaimer: I only tried this with QWENS3.6, dunno if it may work with other models.

Reply to Post

21 Comments

[-]

soshulmedia@reddit

Nice ... but why don't you make a PR upstream? So far, llama.cpp folks seem to be very reasonable, I guess this kind of functionality would be welcome?

[-]

ea_man@reddit (OP)

Good to know, thanks for testing. Well this has nothing to do with llama code base, it's an external extension. I mean it may be integrated in the original UI but I dunno if there's reason for them not to have it already (may be a problem to exclude for models that don't have reasoning).

[-]

soshulmedia@reddit

> Good to know, thanks for testing. Tbh, I didn't test it yet; but I do look forward for someone upstreaming it or something similar into llama.cpp... :D > I mean it may be integrated in the original UI but I dunno if there's reason for them not to have it already (may be a problem to exclude for models that don't have reasoning). Could be, or could not be (I am not affiliated with llama.cpp in any way, just a regular user), but why not try to submit it?

[-]

No_Lingonberry1201@reddit

Not every template defines an enable\_thinking template argument, so this would only work for those that do, which is typically Qwen models. You may need to check the model type.

[-]

ea_man@reddit (OP)

Aye, I declared this for QWEN3.6 only, it should also work with Gemma. I mean it's a quick ack, it runs in the browser.

[-]

No_Lingonberry1201@reddit

Oh cool! I'm definitely liking it!

[-]

CheatCodesOfLife@reddit

Works with Gemma-4, thanks!

[-]

ea_man@reddit (OP)

Nice, good to know, enjoy!

[-]

CheatCodesOfLife@reddit

I made a few violentmonkeys for OpenWebUI a while back, to do things like "move to folder X", "truncate title" (even when the original model is gone)", cycle TTS voice, etc. But they're big clunky buttons positioned in the wrong place. I'll have to study your script to figure out how you placed that so perfectly, then I can properly OpenWebUI (and add the thinking toggle like you've done). I'm waiting for them to update the docker image / fix the fastAPI exploit before I start the instance again though lol.

[-]

ea_man@reddit (OP)

\> I'll have to study your script to figure out how you placed that so perfectly, I opened the web page in the inspector, selected the other button on the side, copy parent element.

[-]

CheatCodesOfLife@reddit

Thanks! I did this (right-clicked the "integrations" button -> inspect element). It's working perfectly with Gemma-4 in OpenWebUI in Firefox (Gemma-4 had to help). https://files.catbox.moe/w5vlbn.png Script here: https://pastebin.com/qhPr3Jqk

[-]

caetydid@reddit

Thank you! I have vibecoded this change every now and then in the svelte webui but hesitated to make a PR to llama.cpp. I still believe this feature needs to be added in the stock server since I dont want to depend on tampermonkey and the likes.

[-]

autoencoder@reddit

This is genius! Thank you! I kept changing the JSON in the dev settings between `{"chat_template_kwargs": {"enable_thinking": false}}` and `{"_chat_template_kwargs": {"enable_thinking": false}}`.

[-]

ea_man@reddit (OP)

That should be what's in the script to: // http://localhost:8080/* I mean reddit filter may fucks things up, I uploaded the original js: [https://store.piffa.net/lm/reasoning\_toggle\_button.js](https://store.piffa.net/lm/reasoning_toggle_button.js)

[-]

autoencoder@reddit

I saw the original. The permissions were for `:8080`, but I removed the colon and port to get it to run. After that, I couldn't work around the `"fetch" is read-only` error in Firefox so I redid the whole script. Mine toggles enable_thinking in the local storage, and adds a refresh button (because llama.cpp assumes it is the only one modifying storage. fair).

[-]

ea_man@reddit (OP)

I think default llama.cp defaults to port 8080, people may have standard web serve on http port 80. Still this come without any documentation yet the stanza: // @match http://localhost:8080/* is where you plan to access the web ui, edit that accordingly if you want it on an other ip or port.

[-]

autoencoder@reddit

Mine also runs on 8080. But the match statement is broken in my Greasemonkey. Whichever way I try to match a port, it disables the script under 'Other user scripts'.

[-]

ea_man@reddit (OP)

Maybe you got some error in the script? As in failed copy / paste or whatever? You could try do dwl the provided file [https://store.piffa.net/lm/reasoning\_toggle\_button.js](https://store.piffa.net/lm/reasoning_toggle_button.js) and see if there's any difference. It could also be a problem in your particular extension release or browser integration I guess, if the web-ui used to work before it should not be network related.

[-]

autoencoder@reddit

It's a bug in either Greasemonkey or Firefox. I doubt your script is the problem; I had trouble with my scripts as well.

[-]

ea_man@reddit (OP)

It's nice when you use the model for quickies like: *tell me the flag for this* *...*and you don't wanna stay there watching generating 4k of reasoning context to get one word. Welcome.

[-]

fake_agent_smith@reddit

One of the maintainers is already working on this [https://github.com/ggml-org/llama.cpp/pull/23434](https://github.com/ggml-org/llama.cpp/pull/23434)