granite-switch-4.1

📝 Read the launch blog: Granite Switch 4.1: Chat, Decisions, and Speech from One Endpoint

Model Summary: Granite Switch 4.1 8B is a modular LLM that composes a base model, multiple adapters and an encoder together so that a single hosted endpoint can move between natural-language responses, general and specialized decisions, and speech-to-text.

Adapters are activated on demand via control tokens in the chat template. For full details on model composition and adapter configuration, see BUILD.md.

• Base Model: ibm-granite/granite-4.1-8b (8B params, 128K context)
• Adapters: 2 — decision (aLoRA, general-purpose Decisions) and guardian-core (aLoRA, from granitelib-guardian-r1.0)
• Audio: embedded speech-to-text via ibm-granite/granite-speech-5.0-470m-turboctc (470M params, English)
• License: Apache 2.0
• Release Date: October 14, 2026
• Backends: HuggingFace Transformers, vLLM
• Automatically Composed with: granite-switch

Granite Switch is also available in granite-switch-4.1-30b (same Chat + Decisions + Guardian + speech configuration, 30B base), and with the complete Granite Libraries bundle (no audio) in granite-switch-4.1-8b-preview, -30b-preview, and -3b-preview.

Included Adapters & Encoders

Speech-to-Text (Audio Input)

Transcription runs on the same endpoint as chat and decisions, so you don't need a separate speech server and how the model is called doesn't change. Incoming audio is transcribed by the embedded Granite Speech 5.0 TurboCTC model (470M params, English, non-autoregressive conformer CTC encoder), the fastest ASR model on the OpenASR Leaderboard . The transcript is spliced into the chat context in place of the audio content part. The speech encoder and the LLM are not yet jointly trained — there is no trained projection coupling them together yet; see docs/AUDIO.md in the granite-switch repo for the full design and limitations (English only, lowercase/unpunctuated output, no decode-time steering).

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")

response = client.chat.completions.create(
    model="ibm-granite/granite-switch-4.1-8b",
    messages=[{"role": "user", "content": [
        {"type": "text", "text": "Answer the question in the audio."},
        {"type": "input_audio", "input_audio": {"data": "", "format": "wav"}},
    ]}],
)
print(response.choices[0].message.content)

Decision Index (JEV) — decision aLoRA

The decision activated-LoRA adapter was evaluated on the public Decision Index fast subset (1,068 rows) in the edition-v0.3 scoring, using the adapter's documented invocation recipe: a fixed decision-engine system prompt, a State: turn (run on the frozen base), and a ### DECISION turn presenting the question and single-token-labelled options, with the first assistant token read as a softmax over the option-label logits. Each configuration was measured two ways — the adapter served locally and the same adapter served behind a hosted vLLM endpoint — which agree to within ~0.6 points, confirming the serving path is faithful; hosted-serving numbers are shown below.

Scores are the Decision Index v0.3 public component (0–100); per-area skills are shown ×100.

Generation

git clone https://github.com/generative-computing/granite-switch.git
cd granite-switch

# Pick what you need:
pip install -e ".[compose]"   # Compose models with adapters
pip install -e ".[hf]"        # HuggingFace inference
pip install -e ".[vllm]"      # vLLM inference
pip install -e ".[audio]"     # Audio input (adds vLLM's audio deps)
pip install -e ".[dev]"       # Everything

Recreating this checkpoint

This checkpoint is produced by composing the base model with guardian-core from the Guardian library and decision from the Core library — --include-adapters drops every other adapter each library ships — plus the ASR cascade (--enable-audio, default Granite Speech 5.0 TurboCTC):

python -m granite_switch.composer.compose_granite_switch \
  --base-model ibm-granite/granite-4.1-8b \
  --adapters ibm-granite/granitelib-guardian-r1.0 ibm-granite/granitelib-core-r1.0 \
  --include-adapters guardian-core decision \
  --enable-audio \
  --output ./granite-switch-4.1-8b

See BUILD.md for the exact adapter ranks/targets this checkpoint ships with, and the adapter catalog for every