Ming Voice runs a local speech-to-speech assistant on a consumer GPU

A home voice assistant can now stay on your own machine. The hardware is a consumer graphics card. Ming Voice is released under the MIT licence by a developer who goes by JMS1717. It pairs inclusionAI's Ming-3.0-audio-tiny model with its own turn-detection and front-end. It answers out loud about a quarter of a second after you stop talking. The repository shows 334 stars six days after it was created on 21 October.

What it does

The project turns Ming-3.0-audio-tiny into a running assistant. The model is a 9.3-billion-parameter speech-to-speech model with 2.7 billion active parameters.

The README names web search through DuckDuckGo or Brave, weather from Open-Meteo, timers, and shopping and to-do lists. It also names a memory of up to 40 facts and control for Home Assistant devices and Jellyfin media players. A wake word mode listens all the time. It answers only speech that starts with the chosen name. Risky actions, such as unlocking a door or opening the garage, are confirmed first.

Audio stays on the machine. The README says only search terms and place names leave the machine when the web tools are on. The user can turn tools off in Settings to stay fully offline.

The latency figures

The project reports its own measurements on a Radeon RX 7900 XTX with 24 GB of memory. End of speech to first audio sits at about 0.23 seconds. The reply is generated during the end-of-turn pause, so it is ready when the pause ends. The language step runs at roughly 115 tokens per second. The first token arrives about 80 milliseconds after a short spoken turn. Speech synthesis runs at about 0.3 times real time, or 13 milliseconds per 80 milliseconds of output audio.

The project also reports a 1.0 percent word error rate when its own speech is transcribed back. It says that matches the reference implementation of the model.

What it costs to try

Ming Voice needs a GPU with 16 GB of memory or more and 30 GB of disk space. It runs on Linux, or on Windows 11 with WSL2. The tested card is the RX 7900 XTX.

Any Radeon RX 6800 or newer card works on Linux with the standard driver that comes with the kernel. The PyTorch install brings its own ROCm 7.2, so nothing system-wide has to change. Nvidia RTX 30-series or newer cards are supported. The project calls that path experimental. It has not yet tested a 16 GB Nvidia card.

The code is MIT. Model weights are the Ming-3.0-audio-tiny weights from inclusionAI, under their own licence. Seven voices are built in. A new voice can be cloned from a 5 to 15 second sample.

A reader who already uses Home Assistant now has an option that does the same job on their own machine. It is not yet an option on a laptop with integrated graphics. The hardware bar is still a graphics card people buy for games.

Source

Ming Voice on GitHub