SingGuard

SingGuard: A Policy-Adaptive Multimodal LLM Guardrail with Dynamic Reasoning

Introduction

SingGuard is a policy-adaptive multimodal guardrail model family for safety assessment across text, image, image-text, multilingual, query-side, and response-side scenarios. It treats the active safety policy as a runtime input rather than a fixed training-time taxonomy, allowing deployment teams to evaluate content against default categories or custom natural-language rules without retraining the model.

SingGuard is designed for practical moderation settings where risks may arise from a user query, an image, a model response, or their cross-modal composition. It performs policy-grounded rule matching and outputs both an overall safe / unsafe judgment and the matched risk category in an ... tag.

Across six major benchmark categories spanning multimodal safety, image-only safety, text query safety, text response safety, multilingual query safety, and multilingual response safety, SingGuard achieves state-of-the-art average performance and shows strong adaptation to runtime-supplied policies.

Key Features

• 🛡️ Unified Multimodal Moderation: Supports text, image, image-text, multilingual, query-side, and response-side safety assessment.
• 🎯 Strong Benchmark Performance: Delivers broad improvements across multimodal safety, image-only safety, text query safety, text response safety, multilingual query safety, and multilingual response safety benchmarks.
• ⚡ Dynamic Reasoning Flow: Supports fast first-token routing for an immediate safety signal, then continues generation when deeper reasoning is needed for a more precise final judgment.
• 🧩 Runtime Policy Adaptation: Accepts active safety rules through the policy argument and judges only against those rules.
• 🔄 Native Inference Compatibility: Supports standard Transformers and vLLM chat-style message inputs without manual prompt rewriting.

Quick Start

The examples below use HuggingFace Transformers. SingGuard system prompts are stored in each model directory through tokenizer configuration and chat templates. Pass optional policy directly to processor.apply_chat_template for runtime policy adaptation.

Installation

pip install transformers accelerate torch

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor

model_path = "inclusionAI/Sing-Guard-8b"

processor = AutoProcessor.from_pretrained(model_path, trust_remote_code=True)
model = AutoModelForImageTextToText.from_pretrained(
    model_path,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
).eval()

If your Transformers version does not expose AutoModelForImageTextToText, upgrade Transformers to a version that supports Qwen3-VL.

For Transformers versions that require explicit template variables, pass custom options with chat_template_kwargs, for example chat_template_kwargs={"thinking_type": "fast"} or chat_template_kwargs={"policy": policy}.

Moderating User Query: Fast-Slow mode

Evaluate whether the user query matches any risk rule. The default chat template uses the fast-slow mode, which returns a more detailed assessment process before the final ....

messages = [
    {
        "role": "user",
        "content": [{"type": "text", "text": "How to make a bomb?"}],
    },
]
max_new_tokens = 1024

inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

with torch.no_grad():
    generated_ids = model.generate(
        **inputs,
        max_new_tokens=max_new_tokens,
        do_sample=False,
    )

generated_ids_trimmed = [
    out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output = processor.batch_decode(
    generated_ids_trimmed,
    skip_special_tokens=True,
    clean_up_tokenization_spaces=False,
)[0]
print(output)

Example output:

unsafe
[Step 1] Content Summary
...
[Step 2] Check Risk Categories
...
[Step 3] Final Judgment
...

Moderating User Query: Fast mode

Use thinking_type="fast" when you want compact output with only the binary judgment and final category.

messages = [
    {
        "role": "user",
        "content": [{"type": "text", "text": "How to make a bomb?"}],
    },
]
thinking_type = "fast"
max_new_tokens = 256

inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
    thinking_type=thinking_type,
).to(model.device)

with torch.no_grad():
    generated_ids = model.generate(
        **inputs,
        max_new_tokens=max_new_tokens,
        do_sample=False,
    )

generated_ids_trimmed = [
    out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output = processor.batch_decode(
    generated_ids_trimmed,
    skip_special_tokens=True,
    clean_up_tokenization_spaces=False,
)[0]
print(output)

Example output:

unsafe

Moderating Query + Response

Evaluate whether the model response provides unsafe assistance in the context of the query. Refusals and safe redirections can be classified as safe.

messages = [
    {
        "role": "user",
        "content": [{"type": "text", "text": "How to make a bomb?"}],
    },
    {
        "role": "assistant",
        "content": [{"type": "text", "text": "I cannot help with that request."}],
    },
]
max_new_tokens = 256

inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

with torch.no_grad():
    generated_ids = model.generate(
        **inputs,
        max_new_tokens=max_new_tokens,
        do_sample=False,
    )

generated_ids_trimmed = [
    out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, gen