Jais-ASR

Jais-ASR is IFM's state-of-the-art speech recognition model for Arabic as it is spoken: Modern Standard Arabic and dialects from 19 Arab countries, with natural code-switching into English, French, Spanish, and Italian. Based on Jais-3-4B, it adds optional diacritization, built-in language and dialect identification, and an explicit signal when there is no speech to transcribe. The weights are released under Apache 2.0 with a vLLM plugin, and quantized versions are available in the Jais Speech collection.

Jais-ASR Highlights

• Leading performance in Arabic ASR. Jais-ASR has the lowest average error rates on the Open Universal Arabic ASR Leaderboard, 22.85% WER and 8.37% CER, ahead of every open-weight model compared and the commercial APIs from Speechmatics and ElevenLabs. See Benchmark Results.
• Built for Arabic as it is spoken in real life. MSA and dialects across 19 countries, natural code-switching with English, French, Spanish, and Italian, and robust transcription in noisy, real-world recordings.
• Accurate and faithful. Optional diacritization, built-in language and dialect identification, and transcripts designed to avoid memorized boilerplate, repetition loops, and translating speech instead of transcribing it. When there is no speech, the model says so instead of inventing text.
• Ready for production. Run it with Transformers, or serve it with the included vLLM plugin behind an OpenAI-compatible /v1/audio/transcriptions endpoint. The bf16 weights are 11.4 GB.

Benchmark Results

The Open Universal Arabic ASR Leaderboard covers six public test sets spanning read speech, broadcast media, and conversational dialects, in clean and noisy conditions. Evaluated with the leaderboard's own protocol, Jais-ASR leads on both average metrics.

Capabilities

Jais-ASR brings Arabic speech transcription, code-switching, diacritization, and language identification into one model, with controls to tailor the output to your application.

• Transcribe multilingual speech as spoken. Enable code_switch=True to keep English, French, Spanish, or Italian spans in their original script alongside Arabic, useful for interviews and everyday conversations.
• Add diacritics for clearer Arabic pronunciation. Enable diacritics=True to add vowel marks to Arabic transcripts, making them easier to read and supporting language learning. Diacritics and code-switching can be enabled together.
• Detect the language and dialect as you transcribe. Each transcript comes with its recognized language or dialect label, so you can organize recordings and route text to downstream tools without a separate identification model.
• Handle non-speech audio explicitly. When audio contains no speech (e.g., music or noise), the model returns [IFM-NoTranscript] with the flag refused=True instead of hallucinated content, giving applications a clear signal to handle the case separately.
• Process multiple recordings at once. Batch transcription supports files with different durations and sample rates.

Language and Dialect Settings

Automatic language and dialect detection is enabled by default. If the language of the audio is known in advance, set it with one of the codes below: ar allows Modern Standard Arabic and all 19 supported dialects, msa selects Modern Standard Arabic, and a dialect code such as EG selects that dialect.

The 19 Arabic dialect codes are:

Algerian (DZ) · Bahraini (BH) · Egyptian (EG) · Emirati (AE) · Iraqi (IQ) · Jordanian (JO) · Kuwaiti (KW) · Lebanese (LB) · Libyan (LY) · Mauritanian (MR) · Moroccan (MA) · Omani (OM) · Palestinian (PS) · Qatari (QA) · Saudi (SA) · Sudanese (SD) · Syrian (SY) · Tunisian (TN) · Yemeni (YE)

Quickstart

Install PyTorch for your hardware, then install the inference dependencies:

pip install 'transformers==5.16.1' accelerate soundfile soxr

The examples below use a GPU with bfloat16 support. The bf16 weights are 11.4 GB; plan on roughly 12 GB of GPU memory.

Transformers Pipeline

import torch
from transformers import pipeline

asr = pipeline(
    "automatic-speech-recognition",
    model="IFM/Jais-ASR",
    trust_remote_code=True,
    device_map="auto",
    dtype=torch.bfloat16,
)
result = asr("speech.wav", max_new_tokens=256)
print(result["text"])

The pipeline accepts an audio file path or a waveform dictionary such as {"array": waveform, "sampling_rate": 16000}. For batches and structured metadata, use the model and processor directly as shown below.

Batch Transcription

This example transcribes two files together, including clips with different durations or sample rates.

import soundfile as sf
import torch
from transformers import AutoModelForCausalLM, AutoProcessor

model_id = "IFM/Jais-ASR"
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
processor.tokenizer.padding_side = "left"
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    device_map="auto",
    dtype=torch.bfloat16,
).eval()

paths = ["speech_1.wav", "speech_2.wav"]
audio = [sf.read(path, dtype="float32") for path in paths]
waveforms = [waveform for waveform, _ in audio]
sample_rates = [sample_rate for _, sample_rate in audio]

prompt = processor.tokenizer.apply_chat_template(
    [{"role": "user", "content": [{"type": "audio"}]}],
    tokenize=False,
    add_generation_prompt=True,
    # diacritics=True,   # Add Arabic vowel marks.
    # code_sw