audio-chat-box

Add the ability to speak with your AI project • hands free • offline • local-first

Speech for a program that talks with people, on the computer it runs on. audio-chat-box reads text aloud with Kokoro, turns speech into text with Parakeet TDT 0.6B v2, and can listen: the client streams its microphone, Silero VAD finds each thing said, and Parakeet hears it once the speaker pauses. It all runs locally, through sherpa-onnx, with nothing sent anywhere.

It is a program of its own that your program starts and talks to over standard input and output, one JSON message a line (docs/protocol.md). It never touches a microphone or a speaker: your program records and plays. It knows nothing of your product's domain: your program tells it which words to expect (hotwords), and it favours them where the sound allows, "knight" over "night" for a chess board, your product's names for yours.

Try it with Docker

The fastest way to try audio-chat-box, with nothing to build: pull the Parrot Service demo from Docker Hub and run it. You speak, it shows what it heard, and it reads it back.

docker pull pnxstudios/audio-chat-box-demo
docker run --rm --platform linux/amd64 -p 127.0.0.1:8080:8080 pnxstudios/audio-chat-box-demo

Then open http://localhost:8080 (that address, not 127.0.0.1), press Record and allow the microphone. You can also type text and hear it in any of Kokoro's 54 voices. Ctrl+C stops it.

The image (about 880 MB to download) carries the program and its models, Kokoro and Parakeet, so it runs with no network, account or API key. It is built for Linux amd64; on Apple Silicon, Docker runs it under emulation, which works but is slower. This is a speech demo, not an LLM chatbot. The Docker demo guide covers building the image yourself, running it with fewer privileges, the models' licences, privacy, browser requirements and limits.

Running it

cargo build --release
target/release/audio-chat-box --models <folder>

It then reads messages, one JSON line each, and answers:

→ {"jsonrpc": "2.0", "id": 1, "method": "initialize", "params": {"protocol": "1.0", "client": {"name": "me", "version": "1"}}}
← {"jsonrpc": "2.0", "id": 1, "result": {"protocol": "1.0", "node": {"name": "audio-chat-box", …}, "capabilities": {"voice": {"speak": true, "transcribe": true, "hotwords": true, "formats": ["wav", "pcmF32"]}}, …}}
← {"jsonrpc": "2.0", "method": "voice/modelState", "params": {"id": "kokoro", "state": {"stage": "ready"}}}
→ {"jsonrpc": "2.0", "id": 2, "method": "voice/speak", "params": {"text": "Your table is ready.", "voice": "am_michael"}}
← {"jsonrpc": "2.0", "id": 2, "result": {"format": "wav", "sampleRate": 24000, "audio": "UklGR…", "durationMs": 1150}}

voice/voices lists Kokoro's 54 voices, voice/models the models and how ready each is, and voice/transcribe hears a WAV file or raw samples, with the hotwords given. On an M1 Max both models load in about a second, and a sentence is read in under one.

voice/listen takes 16 kHz raw samples in voice/audio notifications, as they come from the microphone, and says voice/speechStarted when someone starts speaking and voice/heard with what they said once they pause (800 ms unless the listen says otherwise). One listen at a time. docs/protocol.md has every message.

cargo run --release --example say, hear, listen and bakeoff read text aloud to a WAV file, hear WAV files, listen to WAV files as a stream, and compare transcription models on recordings of your own, for measuring.

The models

The models are downloaded once, the first time each is needed, and each archive is checked against the size and SHA-256 built into the program (src/models.rs) before it is unpacked. By default they come from this repository's models-v1 release: copies of sherpa-onnx's archives, unchanged, and Silero VAD packed the same way, with a NOTICE.txt of their licences. A product may host the same files itself and pass --models-url; a file that differs from the one built in is refused. CREDITS.md says where each came from.

Origins

audio-chat-box began as solrac-voice, the voice of Solrac, an AI-based Chess Coach. Solrac's chat reads its replies aloud and hears its player, by Talk or hands-free, and this program grew out of that work: its models were chosen in Solrac's bake-off of speech recognisers, its listening was tuned on a player's moves said at speed-chess pace, and its protocol is the voice domain of the Chess Node Protocol, which Solrac speaks to all its modules. The comments still cite Solrac's issues (cjus/chessgui#…), as the history of why each choice was made. It was renamed in October 2026, once what was chess in it, the words it favoured, had moved to the client, so that any product can use it.

Licence

Our own code is licensed under either of MIT or Apache-2.0, at your option. Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

The audio-chat-box program as built links sherpa-onnx's prebuilt library, which includes eSpeak NG (GPL-3.0-or-later): Kokoro uses it to sound out words its lexicon doesn't have. So the program as a whole is GPL-3.0-or-later, and its source is this repository with the source of the libraries in sherpa-onnx's prebuilt library: this repository's sources-v1 release, which tools/sources.sh gathers. The models have licences of their own: Kokoro Apache-2.0, Parakeet CC-BY-4.0 (by NVIDIA), Silero VAD MIT.

Every copy of the program goes with the notices of what it is built with: tools/notices.sh writes them to target/licences/THIRD-PARTY-NOTICES.txt. They are the crates' licences (with cargo-about 0.9: cargo install --locked cargo-about@0.9.2 --features cli), then those of the C and C++ libraries in sherpa-onnx's prebuilt library, which the archive doesn't carry, so they are kept in licences/ with where each came from.

Copyright (c) 2026 Carlos Justiniano.