GitHub - pwilkin/openmoss: OpenMOSS pure C++ pipeline based on GGML

Posted by ilintar@reddit | LocalLLaMA | View on Reddit | 8 comments

I'm uploading a full GGML-based pipeline for OpenMOSS (https://huggingface.co/OpenMOSS-Team/MOSS-TTS) that I've vibe-coded for myself in case someone else finds it useful. TTS models are notoriously annoying to set up due to the entire Python ecosystem, so I decided I'd make it a bit simpler. Both server mode and single-shot cli mode are supported here. Why OpenMOSS? For me, the reason was that it's one of the few TTS models that can deal well with languages outside the typical "English/Chinese" duet - namely Polish. Maybe someone else will find it useful as well.

Reply to Post

8 Comments

[-]

ilintar@reddit (OP)

Now per request ( u/unbannedfornothing ) has a WebUI! You can just run it via \`scripts/launch-webui.sh model.gguf\` and it runs the inference server and the WebUI.

[-]

unbannedfornothing@reddit

Thanks, Piotr! Can someone recommend some UI (web\\ windows app) to use in more average user friendly manner?

[-]

ilintar@reddit (OP)

Not yet, but I think I'll add a WebUI.

[-]

pmttyji@reddit

Thanks for this. Prerequisites: * A built llama.cpp tree (`libllama.so`, `libggml*.so`, headers under `ggml/include/` and `include/`). **Build it with the same backend you want here** (`-DGGML_CUDA=ON` for NVIDIA, etc.). That's nice. Though my current laptop has NVIDIA, my upcoming rig gonna have AMD cards so Vulkan/ROCm. Thanks again.

[-]

brahh85@reddit

beyond the hell that python is, there is another hell, using old and non-nvidia hardware in pytorch , because many python TTS engines just ignore it. So your project is a silver line , because it gives support (thanks to ggml) and hardware acceleration to TTS, which is critical for this use case. Thank you so much for your altruism.

[-]

Few_Water_1457@reddit

Legend as always

[-]

caetydid@reddit

thanks for sharing. how much vram does the **MOSS‑TTS‑Realtime** model use? Can I run it bound to a limit? And does it support voice-cloning, too?

[-]

ilintar@reddit (OP)

Realtime is a 1.7B model, so realistically on a Q8 quant it shouldn't use much more than 2GB VRAM.