Kyutai Unmute (incl. TTS) released

[-]

phhusson@reddit

Running unmute locally on RTX3090 adds a bit latency, but it's still a rather fluid conversation. Pretty cool to run locally!

Reply

[-]

harrro@reddit

Did you do the Docker version of the manual version? Also, which LLM model did you use with it (I'm on 3090 as well).

Reply

[-]

I've gotten up to Qwen3-14B AWQ 4-bit, 4k context length on 24GB (4090). Less than 1GB VRAM free with this setup. The whole workload is not very demanding for the GPU, power draw fluctuates, peaks are at about \~130W.

Reply

[-]

Numerous-Aerie-5265@reddit

How did you do this? I’ve been trying for days, always get OOM on the LLM or the tts. Did you just type Qwen/Qwen3-14B-AWQ after “model=“ and what was your “gpu utilization”? I also have a 3090

Reply

[-]

rerri@reddit (OP)

I ran vLLM with this command: vllm serve G:\\booga\\user\_data\\models\\Qwen\_Qwen3-14B-AWQ --max-model-len 2048 --tensor-parallel 1 --chat-template G:\\booga\\user\_data\\models\\Qwen\_Qwen3-14B-AWQ\\qwen3\_nonthinking.jinja --gpu-memory-utilization 0.55 The --gpu-memory-utilization **0.55** was the lowest I could go or vLLM would error when trying to load. That left enough VRAM for unmute to run simultaneously. This is the jinja I was using, it disables thinking which is necessary with a Qwen3 model (unless you want to listen to it think, lol): [https://qwen.readthedocs.io/en/latest/\_downloads/c101120b5bebcc2f12ec504fc93a965e/qwen3\_nonthinking.jinja](https://qwen.readthedocs.io/en/latest/_downloads/c101120b5bebcc2f12ec504fc93a965e/qwen3_nonthinking.jinja)

Reply

[-]

Numerous-Aerie-5265@reddit

Oh that makes sense thank you. But I used docker compose so I guess I’d have to modify that command

Reply

[-]

fractaldesigner@reddit

Hey all — I cloned the [`kyutai-labs/delayed-streams-modeling`](https://github.com/kyutai-labs/delayed-streams-modeling) repo from GitHub, expecting to try out [`unmute.sh`](http://unmute.sh) on their web page but there's no [unmute.sh](http://unmute.sh) in the repo. How do we get this running in windows?

Reply

[-]

rerri@reddit (OP)

First link in OP. You don't need that delayed streams modelling repo to run unmute demo.

Reply

[-]

Old_Paleontologist58@reddit

I am also facing start\_moshi\_server\_public.sh file not found. can you elaborate please. Why is the error occuring and how to fix?

Reply

[-]

rerri@reddit (OP)

The file has "windows line endings" or something. You can tell that to gemini or copilot and ask it how to fix it with dos2unix. I don't know the exact commands anymore, I just followed copilots instructions. Others have had the same issue aswell, someone shared their way of fixing it: [https://github.com/kyutai-labs/unmute/issues/84](https://github.com/kyutai-labs/unmute/issues/84)

Reply

[-]

Independent_Fan_115@reddit

Newbie here. Anyone can share instructions on how to run it locally on a Mac?

Reply

[-]

Organic_Ride547@reddit

Guys anyone knows really good German tts that got a low latency? Most released versions are only on English or and Kyutai now also in French. But I can’t find German ones. Would be great if someone knows something. A low latency would be important too.

Reply

[-]

MustBeSomethingThere@reddit

"To ensure people's voices are only cloned consensually, we do not release the voice embedding model directly. Instead, we provide a [repository of voices](https://huggingface.co/kyutai/tts-voices) based on samples from datasets such as Expresso and VCTK. You can help us add more voices by anonymously [donating your voice](https://unmute.sh/voice-donation)."

Reply

[-]

Hunting-Succcubus@reddit

another DEAD ON ARRIVE.

Reply

[-]

MerePotato@reddit

Dead on arrival for gooners maybe, for the rest of us this is a very useful release

Reply

[-]

Hunting-Succcubus@reddit

there are only rest of gooners.

Reply

[-]

Pedalnomica@reddit

I personally just want something that works and don't really care who it sounds like (unless the voice is like super grating or something). To each their own!

Reply

[-]

Ylsid@reddit

Fuuuuuck offffffff safetycucks

Reply

[-]

fractaldesigner@reddit

im sure there will release this part soon.

Reply

[-]

Hunting-Succcubus@reddit

no Voice Cloning, no interes

Reply

[-]

According_to_Mission@reddit

It has voice cloning I think.

Reply

[-]

Hunting-Succcubus@reddit

not open sourced, so it does not have.

Reply

[-]

JockY@reddit

Better to think before you speak than speak before you think. They withheld the voice cloning model and instead are asking people to donate their voices to create a library.

Reply

[-]

supreme_punk@reddit

It would be really cool if someone could fork this, replacing Unmute's TTS with Chatterbox TTS and it's voice cloning.

Reply

[-]

rerri@reddit (OP)

I don't think Chatterbox supports streaming like Kyutai TTS. From the TTS article: *Kyutai TTS is the first text-to-speech model that is also streaming in text. You can pipe in text as it's being generated by an LLM and Kyutai TTS will already start processing it, leading to ultra-low latency.*

Reply

[-]

supreme_punk@reddit

I think this fork does: [https://github.com/davidbrowne17/chatterbox-streaming](https://github.com/davidbrowne17/chatterbox-streaming)

Reply

[-]

Kindly-Annual-5504@reddit

It's able to steam the response audio yes, but it still needs the full text in order to do that. That's the difference in comparison to this one.

Reply

[-]

supreme_punk@reddit

It could use a smart chunking logic, to start generating the audio gradually. I had made something like that using chat gpt, for a project with chatterbox. Here are the chunking instructions I used (they are a little rough but should give you an idea): When reading text aloud, use commas in short sentences to signal natural pauses and break after them. In longer sentences or lists with many commas, avoid breaking at every comma to prevent choppy speech. Instead, pause only after every second comma. Always break after strong punctuation marks like periods, exclamations, questions, semicolons, colons, and dashes. Short phrases with a single comma should reflect a gentle pause, while long lists should be read fluidly, grouping items rather than listing them one by one.

Reply

[-]

lothariusdark@reddit

>The backbone model is 1B parameters, and the depth transformer is 600M parameters and uses partial weight sharing similar to [Hibiki](https://arxiv.org/abs/2502.03382). > Language(s) (NLP): English and French

Reply

[-]

trararawe@reddit

This would be great for practicing language speech, please add more languages

Reply

[-]

Pedalnomica@reddit

I'm super excited about this! This is exactly what we need for useful voice assistants. I also appreciate them not overselling and admitting there is an actual trade-off putting a text only LLM in the middle, more "intelligence", but it has no idea about things like your tone, etc...

Reply

[-]

harrro@reddit

Yeah I'm surprised at the amount of negativity here because of the voice cloning limitation. If this were just another TTS or STT, then it would be essential but this is basically one of the first good full **realtime** STT-to-LLM-TTS I've seen so far. Looking through their Github, they seem like they've open-sourced all the pieces for the full audio-to-audio pipeline too so I'm definitely going to try to run this locally. The online demo is surprisingly good too. Moshi was a little underwhelming because of the built-in LLM model they had but this seems to allow hooking up any LLM.

Reply

[-]

ShengrenR@reddit

People are just disappointed re the lack of cloning - it was an advertised feature in their original announce and demo, so when it doesn't make it, people are sad. Doesn't mean the rest of it isn't great, but it's still less than what folks had built up in their minds, I suspect.

Reply

[-]

FullOf_Bad_Ideas@reddit

Sweet, I've been waiting for that one. I got it running already, it's pretty nice, latency is low even on single 3090 Ti, though that's with default 1B Gemma model. Model can be swapped out for a different one easily, and that's super powerful. I'll definitely throw a small reasoning LLM at it lol

Reply

[-]

ShengrenR@reddit

give qwen3-30b-a3b a go for the LLM imo - I've not loaded up unmute components to see how much room they eat up, but if there's enough room for the qwen moe it's a good one to use for that super fast response, but still 'smart' enough it's worthwhile

Reply

[-]

mikkel1156@reddit

This seems like a great release. I dont care much for voice cloning, good fast and streamed responses are way more important. I am also using Rust so this might be cool to implement.

Reply

[-]

vamsammy@reddit

Should this work on an M1 Mac?

Reply

[-]

phhusson@reddit

it looks like that's the goal, they pushed MLX for STT and TTS, but for the whole unmute i think it's not there yet.

Reply

Reply to Post

38 Comments