temsai-asr-server

A self-hosted speech-to-text server for NVIDIA Parakeet-TDT models with an OpenAI/Whisper-compatible API. Point any Whisper client at it. It runs on CPU or NVIDIA GPU, on amd64 and arm64.

• POST /v1/audio/transcriptions: same request and response shapes as OpenAI's endpoint (json, verbose_json, text, srt, vtt).
• Per-word timestamps and per-word confidence in verbose_json.
• 25 European languages with automatic language detection (parakeet-tdt-0.6b-v3). An optional English-only model (v2) can be routed for language=en.
• Long audio: switches the encoder to local attention above a threshold, so a 10-minute file fits in memory on CPU.
• Optional noise reduction (RNNoise or DSP spectral gating), off by default: Parakeet is noise-robust on its own (see Noise reduction).
• Optional Bearer-token auth, upload and duration limits.
• Images for CPU, CUDA 12 and CUDA 13 on linux/amd64 and linux/arm64, plus Docker Compose files, a Helm chart and plain Kubernetes manifests.

Why we open-sourced it

Tems.AI turns frontline manufacturing know-how into digital guides, and much of that knowledge arrives as video recorded on the shop floor. Two problems kept coming up with our industrial customers:

• Sovereignty. Many plants cannot send audio from their production floor to a third-party cloud API. Recordings show proprietary processes, the network is often isolated, and data must stay on site or in-country. They need a speech-to-text engine they can run themselves, on their own hardware, with no outbound calls at inference time.
• Noise. Factory audio is hard: motors, conveyors, compressors and HVAC sit right on top of the speech. General-purpose ASR servers transcribe whatever arrives and leave noise handling to the caller.

This server is our answer to both, and we publish it so that anyone with the same constraints can use it:

• It runs fully on premises: CPU or GPU, x86 or ARM, Docker or Kubernetes, including air-gapped clusters (BAKE_MODEL=true puts the model in the image).
• It is built for industrial environments: it is tested against factory noise down to -5 dB SNR, and ships two optional noise-reduction front-ends (RNNoise and DSP spectral gating) with no extra service to deploy.
• It speaks the OpenAI API, so existing Whisper clients and tools switch to a local engine by changing a base URL.

Tems.AI is developed by TemsSoft B.V., which is certified to ISO/IEC 27001 for information security management.

Quick start

Licensing in one line: the source code is Apache-2.0 and free for any use. The official images below are free for evaluation and non-commercial use; running them in production for a business needs a commercial license, sized by the number of production hosts. Images you build yourself are always free. See License.

# CPU (amd64 or arm64). The first start downloads the model (~2.4 GB) into the volume.
docker run -d --name asr -p 8000:8000 -v asr-models:/models ghcr.io/tems-ai/temsai-asr-server:cpu

# NVIDIA GPU (needs the NVIDIA Container Toolkit)
docker run -d --name asr --gpus all -p 8000:8000 -v asr-models:/models ghcr.io/tems-ai/temsai-asr-server:cuda12

curl http://localhost:8000/health          # {"status":"ok","model_loaded":true} once loaded (~1 min on CPU)
curl http://localhost:8000/v1/audio/transcriptions -F file=@meeting.mp3

Any container format ffmpeg can read works: wav, mp3, m4a, flac, ogg, and video files such as mp4 or mov. Audio is extracted automatically.

With the OpenAI SDK

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused-or-your-API_KEY")
with open("meeting.mp3", "rb") as f:
    result = client.audio.transcriptions.create(
        model="nvidia/parakeet-tdt-0.6b-v3",  # any value is accepted, e.g. "whisper-1"
        file=f,
        response_format="verbose_json",
    )
print(result.text)

Image variants

Official images are published to the GitHub Container Registry as ghcr.io/tems-ai/temsai-asr-server, multi-arch, so Docker pulls the right platform:

docker pull ghcr.io/tems-ai/temsai-asr-server:cpu

The CUDA runtime ships inside the PyTorch wheels. The host only needs the driver and the NVIDIA Container Toolkit. Jetson (L4T) is not supported by these images. The CUDA images also run on CPU when no GPU is present (DEVICE=auto).

Build an image yourself (free for any use, including commercial production, under Apache-2.0):

docker build -t temsai-asr-server:cpu .
docker build --build-arg TORCH_VARIANT=cu126 -t temsai-asr-server:cuda12 .
docker build --build-arg TORCH_VARIANT=cu130 -t temsai-asr-server:cuda13 .
docker buildx build --platform linux/arm64 -t temsai-asr-server:cpu-arm64 .
# Air-gapped clusters: bake the default checkpoint into the image (+2.4 GB)
docker build --build-arg BAKE_MODEL=true -t temsai-asr-server:cpu-offline .

Behind a registry mirror, override the base images with --build-arg BASE_IMAGE=... --build-arg FFMPEG_IMAGE=....

Deployment

Docker Compose

docker compose up -d                                         # CPU
docker compose -f compose.yaml -f compose.gpu.yaml up -d     # NVIDIA GPU
API_KEY=change-me docker compose up -d                       # with auth

Kubernetes (Helm)

helm install asr ./deploy/helm/temsai-asr-server                          # CPU
helm install asr ./deploy/helm/temsai-asr-server \
  -f ./deploy/helm/temsai-asr-server/values-gpu.yaml                      # GPU (NVIDIA device plugin required)
helm install asr ./deploy/helm/temsai-asr-server --set apiKey.value=change-me

The chart creates a Deployment, a Service and a PVC for the model cache (kept on uninstall), plus an optional Ingress and Secret. See values.yaml for every option. TORCH_NUM_THREADS follows the container CPU limit by default.

Kubernetes (plain manifests)

kubectl create namespace asr
kubectl apply -n asr -f deploy/kubernetes/cpu.yaml   # or gpu.yaml

These files are rendered from the chart with default values by scripts/render-manifests.sh.

API

POST /v1/audio/transcriptions

multipart/form-data fields:

Other OpenAI fields such as temperature and timestamp_granularities[] are accepted and ignored. verbose_json always contains both words and segments.

Errors: 400 (undecodable or empty audio, bad response_format), 401 (missing or wrong API key), 413 (over MAX_UPLOAD_BYTES or MAX_AUDIO_SECONDS), 415 (non-audio/video content type), 422 (no file).

Example verbose_json:

{
  "task": "transcribe",
  "text": "He hoped there would be stew for dinner, …",
  "language": "en",
  "duration": 10.435,
  "words": [{"word": "He", "start": 0.32, "end": 0.48, "probability": 0.93}, …],
  "segments": [{"id": 0, "start": 0.32, "end": 10.24, "text": "He hoped …", "avg_logprob": -0.21}]
}

Supported languages

parakeet-tdt-0.6b-v3 recognizes these 25 languages (ISO-639-1 codes, as accepted in language and returned in the response):

Bulgarian (bg), Croatian (hr), Czech (cs), Danish (da), Dutch (nl), English (en), Estonian (et), Finnish (fi), French (fr), German (de), Greek (el), Hungarian (hu), Italian (it), Latvian (lv), Lithuanian (lt), Maltese (mt), Polish (pl), Portuguese (pt), Romanian (ro), Russian (ru), Slovak (sk), Slovenian (sl), Spanish (es), Swedish (sv), Ukrainian (uk).

The optional English model (ENGLISH_MODEL_ENABLED) recognizes English only.

Confidence semantics

• words[].probability: NeMo per-word confidence (entropy-based, Tsallis α=0.33, exp-normalized, blank-excluded, min-aggregated over tokens), in [0, 1].
• segments[].avg_logprob: the mean of ln(probability) over the segment's words. It has the same shape as Whisper's (≤ 0, closer to 0 is better) but it is a different estimator, so never compare the numbers across engines.
• If NeMo returns unusable word confidences, probability is omitted and avg_logprob falls back to the length-normalized hypothesis score.
• no_speech_prob is omitted: Parakeet has no no-speech head. Empty text is the no-speech signal. The server never fabricates a value.

Other endpoints

• GET /health: 200 {"status":"ok","model_loaded":true} when ready, 503 while loading. Never requires auth.
• GET /v1/models: the loaded model ids, in OpenAI's list format.

Configuration

All settings are environment variables. Invalid values stop the server at startup with a clear error.

Noise reduction

Parakeet is trained on noisy speech and handles machine noise well without help, so noise reduction is off by default. On benchmark/ (one 214-word passage, as TTS and as a real recording, mixed with pink, machine and industrial noise from 20 to -5 dB SNR) the average WER of parakeet-tdt-0.6b-v3 over 13 conditions was:

Enhancers remove noise a listener hears but leave artifacts the model was not trained on. They can still help on much noisier audio: on an earlier ground-truth factory clip, where Parakeet scored 48% WER, spectral cut it to 32%. Measure on your own recordings with benchmark/noise_benchmark.py before turning DENOISE_ENABLED on.

Sizing

Measured on CPU (4 threads, x86_64) with parakeet-tdt-0.6b-v3:

Measured with full attention, for comparison: 300 s of audio took 58 s, peaked at 9.3 GB and dropped words, and 611 s ran out of memory on a 15 GB host. Local attention handled all of these lengths, which is why the default threshold is 180 s.

Inference is serialized within a container: one request runs at a time and others queue. Scale throughput with replicas. Use MAX_AUDIO_SECONDS together with the memory limit, or split very long recordings on the client side.

Development

python -m venv .venv && . .venv/bin/activate
pip install -r requirements/dev.txt   # no torch/NeMo needed: unit tests stub them
ruff check . && ruff format --check . && pytest

# End-to-end with the real model (builds nothing, uses an existing image):
scripts/smoke_test.sh temsai-asr-server:cpu

CI (.github/workflows/) runs lint, unit tests, Helm/manifest validation and hadolint on every push and PR. Image builds are smoke-tested with the real model:

• pushes to main and PRs that touch the image build only the CPU linux/amd64 image and publish nothing;
• a release tag vX.Y.Z builds all six variants on native amd64 and arm64 runners and publishes multi-arch manifests to GHCR;
• Run workflow (manual) builds all six without publishing.

Community

Contributions are welcome. Pull requests are accepted under the contribution terms: Apache-2.0, signed off with git commit -s, and usable in the official images.

• New here? Look for issues labelled good first issue and read CONTRIBUTING.md.
• Bugs and feature requests: open an issue.
• Security problems: report them privately as described in SECURITY.md.
• Everyone taking part follows the Code of Conduct.
• Accessibility: what the captions can and cannot do, and how to report a barrier, in ACCESSIBILITY.md.
• Changes per release are listed in CHANGELOG.md.
• Using an AI coding agent? AGENTS.md (also CLAUDE.md) has the project's commands and rules.

License

Source code: Apache-2.0. You can use, modify and redistribute the code, and run images you build from it, for any purpose, commercial included, free of charge.

Official images: commercial license. The images Tems.AI publishes at ghcr.io/tems-ai/temsai-asr-server, from version 0.2.0 on:

The commercial license is a subscription sized by the maximum number of production hosts: the nodes of each production Kubernetes cluster that runs the images (or only the dedicated node pool, if you pin the images to one), or the Docker hosts that run them. Details are in COMMERCIAL-LICENSE.md. For pricing and licenses, write to support@tems.ai. Licenses are granted by TemsSoft B.V., the ISO/IEC 27001-certified company that develops Tems.AI.

What the license pays for is the maintained build: six tested image variants (CPU, CUDA 12 and CUDA 13 on amd64 and arm64), pinned NeMo and PyTorch versions, vulnerability scanning and patched dependencies, and support. If you would rather maintain your own build, docker build . gives you the same server under Apache-2.0. Images released before 0.2.0 stay Apache-2.0.

Third-party components keep their own licenses, listed in NOTICE and copied into every image under /licenses. The model weights are not part of this repository and are downloaded from Hugging Face at runtime: nvidia/parakeet-tdt-0.6b-v3 and -v2 are licensed by NVIDIA under CC BY 4.0. If you redistribute an image built with BAKE_MODEL=true, it contains those weights, so keep the attribution from NOTICE.

Images derived from the official images (built FROM them, or modified, mirrored or re-tagged copies) stay under the commercial license, and if you publish one it must be distributed under that license too. Images you build yourself from source are Apache-2.0 and may be published, but not under the Tems.AI name: "Tems.AI" is a trademark of TemsSoft B.V., and a public self-built image must not use "Tems.AI", "TemsAI" or "temsai" in its image name, tags or vendor label.

This project is not affiliated with or endorsed by NVIDIA. "NVIDIA" and "Parakeet" are used only to identify the models this server runs.