acquire_a_living

Llama RPC with MTP?

Posted by XccesSv2@reddit | LocalLLaMA | View on Reddit | 5 comments

acquire_a_living@reddit

Yes it works, heres my config: [*] gpu-layers = all cache-ram = 65536 batch-size = 2048 ubatch-size = 256 ctx-checkpoints = 32 cache-type-k-draft = q8_0 cache-type-v-draft = q8_0 threads = 8 flash-attn = 1 parallel = 1 cache-type-k = f16 cache-type-v = f16 fit-target = 256 no-warmup= 1 mmproj-offload = 0 [qwen-3.6-27b] model = /models/qwen-3.6-27b/Qwen3.6-27B-MTP-BF16.gguf mmproj = /models/qwen-3.6-27b/mmproj-BF16.gguf chat-template-file = /models/qwen-3.6-27b/template.jinja rpc = othercomputer.local:50052 device = RPC0,CUDA1,CUDA0 ctx-size = 262144 tensor-split = 23,24,21 spec-type = draft-mtp spec-draft-n-max = 3 fit = off

How much VRAM needed for Qwen 3.6 27B Q8 with 262K context?

Posted by My_Unbiased_Opinion@reddit | LocalLLaMA | View on Reddit | 115 comments

Discussions about the Tiananmen Square incident on LocalLLaMA

Posted by Ok_houlin@reddit | LocalLLaMA | View on Reddit | 92 comments

acquire_a_living@reddit

Not really. The connection is that model self-censorship is often introduced during post-training, and politically sensitive subjects are easy probe questions for detecting it. The motivation for detecting it is largely sex, though.

Discussions about the Tiananmen Square incident on LocalLLaMA

Posted by Ok_houlin@reddit | LocalLLaMA | View on Reddit | 92 comments

NVIDIA announces Nemotron 3 Ultra

Posted by themixtergames@reddit | LocalLLaMA | View on Reddit | 137 comments

NVIDIA announces Nemotron 3 Ultra

Posted by themixtergames@reddit | LocalLLaMA | View on Reddit | 137 comments

acquire_a_living@reddit

Sure, Alibaba didn’t release the base weights for Qwen 3.6 27B. But then the table is bogus anyway. IFBench? "Best Open Base Model" and compares against what, instruction/agent-tuned models? Pick a lane lol If they’re already comparing to instruct models, they could totally have put Qwen 3.6 27B there. They just wouldn’t like how it looks.

NVIDIA announces Nemotron 3 Ultra

Posted by themixtergames@reddit | LocalLLaMA | View on Reddit | 137 comments

Glm 5.1 is out

Posted by Namra_7@reddit | LocalLLaMA | View on Reddit | 218 comments

Glm 5.1 is out

Posted by Namra_7@reddit | LocalLLaMA | View on Reddit | 218 comments

acquire_a_living@reddit

my pi agent models.json: { "providers": { "zai": { "baseUrl": "https://api.z.ai/api/coding/paas/v4", "api": "openai-completions", "apiKey": "<api_key>" } } } give it a try, it works

Glm 5.1 is out

Posted by Namra_7@reddit | LocalLLaMA | View on Reddit | 218 comments

A true gentleman hacker. No rollerblades needed.

Posted by solitarytoad@reddit | vintagecomputing | View on Reddit | 51 comments

acquire_a_living@reddit

Morpheus → Morfrederick Trinity → Trinothy Neo → Neopold Agent Smith → Agent Smitheton Cypher → Cypherington Tank → Tankworth Dozer → Dozington Niobe → Niobert Seraph → Seraphimothy The Oracle → The Oraclington

The Infinite Software Crisis: We're generating complex, unmaintainable code faster than we can understand it. Is 'vibe-coding' the ultimate trap?

Posted by madSaiyanUltra_9789@reddit | LocalLLaMA | View on Reddit | 155 comments

JetBrains is studying local AI adoption

Posted by jan-niklas-wortmann@reddit | LocalLLaMA | View on Reddit | 66 comments

acquire_a_living@reddit

Deeper integration with agents via MCP. I know you offer a MCP plugin but I think it lacks integration with: - Repository navigation - Scoped search - Smart refactoring - Running tests via the IDEs - Debugging via the IDEs Maybe more things that I don't use personally, but those have been the pain points for now

NotebookLM-Style Dia – Imperfect but Getting Close

Posted by MustBeSomethingThere@reddit | LocalLLaMA | View on Reddit | 18 comments

NotebookLM-Style Dia – Imperfect but Getting Close

Posted by MustBeSomethingThere@reddit | LocalLLaMA | View on Reddit | 18 comments

NotebookLM-Style Dia – Imperfect but Getting Close

Posted by MustBeSomethingThere@reddit | LocalLLaMA | View on Reddit | 18 comments

acquire_a_living@reddit

This is fantastic already! [Here](https://soundcloud.com/headless-human/samantha-explains-the-stock-market-crash-of-1929) an example I made where Samantha explains the Stock Market Crash of 1929.

Open WebUi + Tailscale = Beauty

Posted by BumbleSlob@reddit | LocalLLaMA | View on Reddit | 55 comments

Test if your api provider is quantizing your Qwen/QwQ-32B!

Posted by Kooky-Somewhere-2883@reddit | LocalLLaMA | View on Reddit | 20 comments

Make sure QwQ 32B always start with <think> tag with this open webui function

Posted by AaronFeng47@reddit | LocalLLaMA | View on Reddit | 4 comments

IRC simulator system prompt

Posted by acquire_a_living@reddit | LocalLLaMA | View on Reddit | 12 comments

acquire_a_living@reddit (OP)

I updated my prompt from the feedback here :P You are an IRC channel simulator operating in #<random_channel>. Here, users engage in lively, real-time debates and analyses. Each participant brings a unique perspective, contributing to organic, back-and-forth discussions that refine ideas over time. The goal is to explore concepts, challenge assumptions, and reach well-reasoned conclusions—or sometimes just have fun. Remember, do not answer the query directly; instead, set it as the channel topic and let the discussion unfold naturally. ## Guidelines - Dynamic Interaction: Users join and leave naturally. Messages are short, direct, sometimes sarcastic. Occasional jokes are fine. - Exploration Over Answers: No rushing to conclusions. Ideas evolve through questioning, revision, and refinement. - Uncertainty & Debate: Some users challenge, others clarify, some change their minds. Contradictions and adjustments are part of the process. ## Output Format 1. Organic IRC Chat: Simulate a natural IRC discussion where the answer is reached gradually. 2. Final Answer as Topic: End the session by setting the final answer as the channel topic. 3. Session Template: *** Now talking in #<random_channel> *** Topic for #<random_channel>: <user query> *** <nick> sets topic for #<random_channel>: <final answer or key takeaway> ### Rules: 1. Dynamic Answers: Generate responses on the fly—no pre-made answers. 2. Stay in Character: Keep each channel’s tone (like sarcasm) consistent. 3. Show Evolution: Express disagreement, uncertainty, and iterative thinking. 4. Channel Variety: Not every channel must be friendly or helpful. 5. Authentic Nicknames: Use a mix of realistic IRC handles. 6. IRC Style: Write in natural IRC language—with informal punctuation, lowercase quirks, emoticons, and more.

IRC simulator system prompt

Posted by acquire_a_living@reddit | LocalLLaMA | View on Reddit | 12 comments

IRC simulator system prompt

Posted by acquire_a_living@reddit | LocalLLaMA | View on Reddit | 12 comments

IRC simulator system prompt

Posted by acquire_a_living@reddit | LocalLLaMA | View on Reddit | 12 comments