Nemotron-3.5-Rerank

Description:

Nemotron-3.5-Rerank-8B-BF16 was developed by NVIDIA for multilingual and multimodal question-answering retrieval. It is a world-class reranker optimized to produce a logit representing the relevance of a document to a text query. Documents may contain text, page images, or both, including charts, tables, and infographics. The model can be integrated into classical or agentic vision RAG pipelines. For text retrieval, the model was evaluated across 34 languages: English, Arabic, Assamese, Bengali, Bulgarian, Chinese, Danish, Dutch, Finnish, French, German, Hindi, Hinglish, Indonesian, Italian, Japanese, Korean, Malay, Marathi, Nepali, Norwegian, Persian, Portuguese, Romanian, Russian, Spanish, Swahili, Swedish, Tamil, Telugu, Thai, Ukrainian, Urdu, and Vietnamese. For image retrieval, it was evaluated across 18 languages: Arabic, Bengali, Chinese, English, Persian, Finnish, French, German, Hindi, Indonesian, Japanese, Korean, Russian, Spanish, Swahili, Telugu, Thai, and Yoruba. The model may support additional languages beyond these evaluation sets.

This model is ready for commercial or non-commercial use.

License/Terms of Use:

Governing Download Terms: Use of the model is governed by the OpenMDW License Agreement, version 1.1 (OpenMDW-1.1). Additional Information: The Ministral-3-8B-Instruct-2512-BF16 base model is governed by the Apache License, Version 2.0.

Deployment Geography:

Global

Use Case:

The reranking model serves as a key component of a multimodal retrieval system, such as a vision RAG pipeline, where it helps improve overall retrieval accuracy. A multimodal retrieval system typically uses a multimodal embedding model (dense retrieval) to return a set of potentially relevant documents given an input query. The reranking model then evaluates these candidate documents in greater detail and reorders them based on their relevance to the query, producing a final ranking. By refining the initial retrieval results, the reranking stage can improve the precision of the retrieved documents and ultimately lead to more accurate downstream responses.

Release Date:

Hugging Face 10/20/2026 via https://huggingface.co/nvidia/Nemotron-3.5-Rerank-8B-BF16

Model Architecture:

Architecture Type: Transformer Network Architecture: The released model contains a Pixtral-based vision encoder and a bidirectional Ministral3 language-model tower. Training proceeded in two stages. In the first stage, an 8B reranker checkpoint derived from the Ministral-3-8B-Instruct-2512-BF16 backbone was trained on a blend of text-only datasets. In the second stage, the 8B reranker model initialized from the first-stage checkpoint was trained on image and combined image-text datasets.

Number of Model Parameters: 8.38B

Input:

Input Type(s): Text, Image Input Format(s): Text: String; Image: Red, Green, Blue (RGB) Input Parameters: Text: One-Dimensional (1D), Image: Two-Dimensional (2D) Other Properties Related to Input: Each input is one text query paired with one document. The document may be an image, text, or a combination of image and text. The default image longest_edge is 1,120 pixels and can be increased to 1,540. The maximum context length is 32,768 tokens.

Output:

Output Type(s): Floats Output Format: List of Floats Output Parameters: One-Dimensional (1D) Other Properties Related to Output: Each output value corresponds to a raw logit. The higher the value, the more relevant a document is to a given query.

Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA's hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.

Installation

Create a Python 3.12 environment for the Transformers and Sentence Transformers examples:

uv venv --python 3.12 .venv-transformers
source .venv-transformers/bin/activate
uv pip install torch "transformers>=5.5.0,<5.17.0" accelerate pillow packaging ninja
uv pip install flash-attn --no-build-isolation

The examples use Flash Attention 2. The Transformers upper bound avoids the Pixtral Flash Attention regression. The checkpoint uses TokenizersBackend and does not require mistral_common.

Model training and evaluations used transformers==5.12.1. Example outputs can vary with hardware and runtime.

Transformers Usage

Score query/document pairs containing text, images, or both.

import torch
from transformers import AutoModelForSequenceClassification, AutoProcessor
from transformers.image_utils import load_image

MODEL_PATH = "nvidia/Nemotron-3.5-Rerank-8B-BF16"  # Or a local checkpoint directory.
IMAGE_URL = "https://docs.nvidia.com/nemo-framework/user-guide/24.07/_images/nemo-llm-mm-stack.png"
MAX_MODEL_LEN = 32768
DEVICE = "cuda:0"

def main() -> None:
    if not torch.cuda.is_available():
        raise RuntimeError("This bfloat16 example requires a CUDA GPU.")

    processor = AutoProcessor.from_pretrained(MODEL_PATH)

    model = AutoModelForSequenceClassification.from_pretrained(
        MODEL_PATH,
        trust_remote_code=True,
        dtype=torch.bfloat16,
        device_map={"": DEVICE},
        attn_implementation="flash_attention_2",
    ).eval()

    def rerank(
        query: str,
        documents: list[dict],
    ) -> torch.Tensor:
        scores = []
        for document in documents:
            document_content = []
            if document.get("text") is not None:
                document_content.append({"type": "text", "text": document["text"]})
            if document.get("image") is not None:
                document_content.append({"type": "image", "i