nemotron-labs-unified-asr

Nemotron-Labs-Unified-ASR-0.6b is the multilingual version of Parakeet-unified-en-0.6b — a unified automatic speech recognition (ASR) model that combines offline and streaming inference (with a minimum algorithmic latency of 160ms) in a single model [1]. It extends the English-only unified model to 25 European languages, and is trained primarily on the Granary dataset [5], [6] using the same training data composition as parakeet-tdt-0.6b-v3. The model transcribes speech with punctuation and capitalization support, and uses language-ID prompt conditioning on the encoder to select the target language at inference time.

In our evaluation setup, the new unified model outperforms parakeet-tdt-0.6b-v3 in offline mode and nemotron-3.5-asr-streaming-0.6b in most streaming scenarios with algorithmic latency down to 160ms (see Comparison with already released models).

The new unified model also serves as a strong ASR front end for cascaded streaming speech translation (see Streaming Speech Translation).

Key capabilities:

• One multilingual checkpoint for both tasks: a single unified model covers both offline and streaming inference with a minimum latency of 160ms across 25 European languages.
• Language-ID prompting: the target language is passed to the encoder as a prompt, which stabilizes language selection in low-latency streaming. An automatic language mode (target_lang=auto) is also supported.
• Streaming chunk size flexibility: the streaming latency (chunk + right context) is configurable from 2080ms down to 160ms in steps of 80ms, without re-training.
• Punctuation & Capitalization: built-in support for punctuation and capitalization in the output text.
• Speech translation front end: the right context lets the current chunk use upcoming audio before the transcript is committed, which gives the LLM of a cascaded speech translation system a more accurate source transcript.

Supported Languages: Bulgarian (bg), Croatian (hr), Czech (cs), Danish (da), Dutch (nl), English (en), Estonian (et), Finnish (fi), French (fr), German (de), Greek (el), Hungarian (hu), Italian (it), Latvian (lv), Lithuanian (lt), Maltese (mt), Polish (pl), Portuguese (pt), Romanian (ro), Slovak (sk), Slovenian (sl), Spanish (es), Swedish (sv), Russian (ru), Ukrainian (uk)

License/Terms of Use:

GOVERNING TERMS: Use of this model is governed by the OpenMDW License Agreement, Version 1.1.

This model is ready for commercial/non-commercial use.

Deployment Geography:

Global

Use Case:

This model can be used to transcribe multilingual speech (25 European languages) into text using offline or streaming inference with latency as low as 160ms. It supports a variety of applications, such as voice assistants, live captioning, conversational bots, subtitle generation, and, combined with an LLM translator, streaming speech translation.

Release Date:

Hugging Face 10/27/26

Model Architecture:

Architecture Type: Unified-FastConformer-RNNT with language-ID prompt [1], [2] — an optimized version of the Conformer model [4] with 8x depthwise-separable convolutional downsampling and RNNT loss.

Network Architecture:

• Encoder: Unified FastConformer with 24 layers
• Decoder: RNNT (Recurrent Neural Network Transducer)
• Language conditioning: language-ID prompt fused into the encoder output
• Parameters: 600M

Model Initialization

The encoder was initialized from the parakeet-tdt-0.6b-v3 model to give the unified model strong multilingual acoustic representations from the beginning of training.

Unified offline/streaming training

The model was trained jointly in offline and streaming modes. In the offline mode we use standard offline training with full-context self-attention and non-causal convolutions. In the streaming mode we apply chunked self-attention masks (left, middle/chunk and right context) together with Dynamic Chunked Convolutions inside each FastConformer layer [3] to adapt the model to both decoding scenarios. We also use a mode-consistency regularization (MCR) loss [1] to further reduce the gap between offline and streaming performance. All model parameters are shared between offline and streaming modes (encoder, predictor, and joint networks), including the initial x8 subsampling with non-causal convolutions.

Language-ID prompting

The model uses language-ID prompt conditioning on the encoder, following the approach used in nemotron-3.5-asr-streaming-0.6b. The language prompt is fused with the acoustic representation as follows:

• The FastConformer encoder processes audio into an acoustic embedding of shape (D, T).
• The language encoding expands a one-hot language vector across the time axis → (K, T), broadcasting the language identity to every frame.
• Concatenation along the feature axis produces a fused tensor (D + K, T).
• A projection layer maps the fused features to the RNNT decoder.

Why the prompt is needed. The model uses a single shared SentencePiece tokenizer built from the combined transcripts of all 25 languages, so the tokenizer itself carries no language information and the output token space is shared across languages. In low-latency streaming configurations (latency ≈ chunk + right con