Is anyone talking verbally to their models and have them talking back through TTS?

Posted by Borkato@reddit | LocalLLaMA | View on Reddit | 17 comments

Wondering what the easiest OSS setup for this is on 24gb ram, or if I have to cobble things together out of parakeet and ooba or something else? I just got a new computer and I’m growing tired of all the setup and tinkering, but I know it’s worth it 💀

Reply to Post

17 Comments

[-]

Brief-Common-1673@reddit

If you want to add your phone to the mix, you can use Dialog (1). It's a simple and well architected STT-TTS orchestration layer. Please feel free to reach out to the author if you need help with a deployment. Currently it has concrete TTS implementations for both Cartesia and 11Labs - hence, there is a wide selection of voices. For STT there are implementations for Deepgram and OpenAI. (1) [https://github.com/faranalytics/dialog/tree/main](https://github.com/faranalytics/dialog/tree/main)

[-]

ComplexTechnician@reddit

I was pretty stunned at how well open-webui did with this out of the box. I’m on the road a lot so I do STT->LLM->TTS probably 90+% of the time. I wanna replace it with a better TTS but overall it’s been how I interface with gpt oss 20b

[-]

atineiatte@reddit

chatterbox-tts. I'm waiting for them to sort out the issues with multilingual so I can use it to practice Russian no one asked, but I think setting up language models for voice conversations is a really slippery slope

[-]

ReMeDyIII@reddit

There's a game called Whispers from the Stars that surprised me at how amazing it is at voice communication, and the responses are almost fast enough that it feels like a natural conversation. Now as far as getting it on a 24 GB setup though... I remain unconvinced.

[-]

Borkato@reddit (OP)

What kind of slippery slope? Also thanks! I’ll check it out :)

[-]

dazzou5ouh@reddit

Watch the movie HER

[-]

dazzou5ouh@reddit

HER

[-]

AppearanceHeavy6724@reddit

> to practice Russian > HER Lol

[-]

swagonflyyyy@reddit

Yes. I do it everyday.

[-]

teachersecret@reddit

I built a simple little system with parakeet->llm->kokoro. You can fit something like oss 20b or one of the mistral smalls in there alongside kokoro and parakeet and stand it all up in a single docker in less than 24gb vram with near instant response. It works, but I’ve found I don’t like verbally talking to the computer. It’s a little weird chatting aloud to it, and if you have others around you, it’s even weirder.

[-]

mumblerit@reddit

theres a faster whisper docker container and kokoro fastapi docker container that work ok first the LLM has to generate the text, then the tts generates the reply, so it will add some latency

[-]

Professional-Bear857@reddit

I'm running openweb UI, that has speech to text and with a bit of configuring text to speech as well, although the audio could be better

[-]

QuantuisBenignus@reddit

For Linux, I extended [this](https://github.com/QuantiusBenignus/BlahST) speech-to-text input tool into a low-resource Speech-to-Speech Chat (llama.cpp based): [BlahST - Speech Input in Any Editable Text Field](https://github.com/QuantiusBenignus/BlahST) Multilingual demonstration (please, turn on the sound for this and the other demo videos): [Multilingual Interactive Speech Chat with blahstbot](https://github.com/QuantiusBenignus/BlahST/blob/main/SPEECH-CHAT.md) WIP. (Need to root out some brittleness in the streaming conversation `blahstream`) but promising speed and low latency due to pythonless implementation (zsh orchestrator).

[-]

Borkato@reddit (OP)

Ooh, this is awesome 👀

[-]

QuantuisBenignus@reddit

May require a bit of setup and tinkering, but the README should help.

[-]

davidpfarrell@reddit

Do you use an instruction to help the model keeps its replies short and more tts-friendly?

[-]

Borkato@reddit (OP)

I don’t have TTS set up yet, so I don’t know, but yes I would! I’m also wondering if I could set up something like accents for characters, maybe even code something that would cause it to do it separately from narration. Would be neat if it read like a story!