d1-omni

src="https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/2b08LKpev0DNEk6DlnWkY.png" alt="Liquid AI" style="width: 100%; max-width: 100%; height: auto; display: inline-block; margin-bottom: 0.5em; margin-top: 0.5em;" />

d1-omni-600M

d1-omni-600M is a 600M parameter decision model built on LFM2.5-Encoder-350M. You give it a state (text or JSON, with images or a voice clip) and a set of named questions. It returns typed answers with zero output tokens: every answer is read directly from the model's distribution over the options, with no generation and no parsing.

• Vision-language: text and images (tiled for large frames, several images per state) in a single forward pass.
• Audio-language: text and up to 30 s of speech in a single forward pass.
• Edge-sized: 587M parameters: a 381M shared trunk and decision head, a 94M vision encoder and a 112M audio encoder. Every modality runs the same trunk weights.

Find more information about open d1 in our blog post.

🗒️ Model Details

• Total parameters: 587M
• Vision encoder: SigLIP2 vision tower from LFM2.5-VL-450M
• Audio encoder: 17-layer FastConformer
• Context length: 16,384 tokens (text, image and audio positions together); with images, the state and question text is cut to 896 tokens, as trained
• Vocabulary size: 65,536

We recommend d1-omni-600M wherever a pipeline needs a yes/no, a pick from named options, or a rating: routing and triage, moderation, intent and topic classification, voice-command routing, extraction checks, reranking, agent guardrails, and visual inspection. It is not a chat model and does not write text.

[!WARNING] ⚠️ Audio capabilities were trained on requests between an English speaker and an assistant. Training tasks include what kind of utterance it is, its topic, and what the speaker wants. Clips are also cut at 30 seconds.

🏃 How to use

Install the dependencies (requires transformers>=5.15):

pip install "transformers>=5.15" torch torchvision pillow soundfile

The model ships its own code, so load it with trust_remote_code=True:

import io
from urllib.request import urlopen

import soundfile as sf
import torch
from transformers import AutoModel
from transformers.image_utils import load_image

device = "cuda" if torch.cuda.is_available() else "mps" if torch.backends.mps.is_available() else "cpu"
dtype = torch.float32 if device == "cpu" else torch.float16
model = AutoModel.from_pretrained("LiquidAI/d1-omni-600M", trust_remote_code=True, dtype=dtype).to(device)

# Text: several named questions over one state, answered in one pass
questions = {
    "refund": {
        "type": "noul",
        "instructions": "Is the customer asking for a refund?",
    },
    "team": {
        "type": "choice",
        "instructions": "Which team should handle this?",
        "criteria": {
            "billing": "Charges, refunds, invoices",
            "technical": "App or site faults",
            "fraud": "Suspected unauthorised use",
        },
    },
    "urgency": {
        "type": "score",
        "instructions": "How urgent is this?",
        "criteria": ["Can wait", "Today", "Blocking the customer now"],
    },
}
print(model.system_one("I was charged twice this month, please refund one of them.", questions))

# Image: the photo is the whole state
image = load_image("http://images.cocodataset.org/val2017/000000039769.jpg")  # two cats on a sofa
cats = {
    "type": "choice",
    "instructions": "How many cats are there?",
    "criteria": {"one": "One", "two": "Two", "more": "Three or more"},
}
print(model.system_one(None, {"cats": cats}, images=[image]))

# Text + audio: one 16 kHz mono clip
url = "https://huggingface.co/datasets/Narsil/asr_dummy/resolve/main/1.flac"
audio, rate = sf.read(io.BytesIO(urlopen(url).read()), dtype="int16")
topic = {
    "type": "choice",
    "instructions": "What is the speaker talking about?",
    "criteria": {"food": "Food and meals", "travel": "Travel and transport", "weather": "The weather"},
}
print(model.system_one("Voice note from a user.", {"topic": topic}, audio=audio))

# Batch: many requests in one call
tickets = ["Where is my parcel? It was due Monday.", "The app crashes when I open settings."]
print(model.system_one_batch([(t, {"team": questions["team"]}) for t in tickets]))

A state is a string, any JSON value, or None when the images or the audio are the whole state. images is a PIL image or a list of them; audio is one 16 kHz mono clip as int16 or float samples. A request carries images or audio, not both: passing both raises a ValueError.

The model was trained in float32. On GPUs, float16 is faster and keeps the float32 answers: on our checks it gave the same top answer on every text (243), image (214) and audio (416) row. Avoid bfloat16, which changed the top answer on 0.8% of text and 1.7% of audio rows.

Questions and answers

Questions follow the Decision Index schema: type, instructions, and criteria.