llama-nv-embed-reasoning

Model Overview

Description

llama-nv-embed-reasoning-3b is a 3.2B-parameter embedding model designed to produce high‑quality sentence and document representations for retrieval, semantic search, and similarity tasks, with a strong focus on reasoning‑heavy content.

Built on a Llama‑style encoder and trained with contrastive objectives on diverse text (including question–answer pairs, technical explanations, and multi‑step reasoning data), the model is optimized to:

• Capture deeper logical and semantic relationships beyond surface keyword overlap
• Align short queries with long, information‑dense documents
• Support retrieval for tasks involving explanations, step‑by‑step reasoning, and problem solving

The model outputs dense vector embeddings suitable for use with standard vector databases and retrieval pipelines. Its 3B size offers a balance between quality and inference efficiency, making it suitable for both experimentation and latency‑sensitive workloads.

This model is for non-commercial/research use only.

License/Terms of Use

The use of this model is governed by the Creative Commons Non-Commercial License. The model is built with meta-llama/Llama-3.2-3B which is released under Llama 3.2 Community License Agreement.

This project will download and install additional third-party open source software projects. Review the license terms of these open source projects before use.

Deployment Geography

Global

Use Case

llama-nv-embed-reasoning-3b is a compact, accurate embedding model intended for practitioners building applications that retrieve and organize information based on deep semantic and reasoning relationships rather than simple keyword overlap. It is instrumental in text‑only RAG systems, where both queries and documents may contain multi‑step arguments, explanations, or technical details, and where aligning concise questions with long, information‑dense passages is critical. Potential applications include reasoning‑aware RAG for assistants and chatbots, semantic search over technical or knowledge‑base content, intelligent document and ticket retrieval, clustering and deduplication of related texts, and analytics pipelines that require robust similarity signals between questions, rationales, and answers.

Release Date

03/10/2026 via https://huggingface.co/nvidia/llama-nv-embed-reasoning-3b

Model Version(s)

llama-nv-embed-reasoning-3b version 1.

Model Architecture

• Architecture Type: Transformer
• Network Architecture: meta-llama/Llama-3.2-3B

The llama-nv-embed-reasoning-3b is a transformer-based text embedding model built from meta-llama/Llama-3.2-3B. It has approximately 3.2B parameters.

Input(s)

Input type(s): Text

Input format(s): A list of text strings.

Input parameter: One-Dimensional (1D)

Other properties related to input: The model was trained with a maximum token length of 512 tokens for both queries and passages, while evaluation used a maximum token length of 8192 tokens.

Output(s)

Output type(s): Floats

Output format(s): A list of float arrays.

Output parameter: One-Dimensional (1D)

Other properties related to output: The model outputs an embedding vector with up to 3072 dimensions for each input text string.

Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA’s hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.

Getting Started

Installation

The model requires transformers version 4.51.0 and flash-attention installed.

pip install transformers==4.51.0
pip install flash-attn==2.6.3 --no-build-isolation
pip install accelerate==0.34.2

Usage

Hugging Face Transformers

import torch
import torch.nn.functional as F
from transformers import AutoTokenizer, AutoModel

def average_pool(last_hidden_states, attention_mask):
    """Average pooling with attention mask."""
    last_hidden_states_masked = last_hidden_states.masked_fill(~attention_mask[..., None].bool(), 0.0)
    embedding = last_hidden_states_masked.sum(dim=1) / attention_mask.sum(dim=1)[..., None]
    embedding = F.normalize(embedding, dim=-1)
    return embedding

model_name = "nvidia/llama-nv-embed-reasoning-3b"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name, trust_remote_code=True)
model = model.to("cuda:0")
model.eval()
query_prefix = "query:"
document_prefix = "passage:"

queries = [
    "how much protein should a female eat",
    "summit define",
]
documents = [
    "As a general guideline, the CDC's average requirement of protein for women ages 19 to 70 is 46 grams per day. But, as you can see from this chart, you'll need to increase that if you're expecting or training for a marathon. Check out the chart below to see how much protein you should be eating each day.",
    "Definition of summit for English Language Learners. : 1  the highest point of a mountain : the top of a mountain. : 2  the highest level. : 3  a meeting or series of meetings between the leaders of two or more governments."
]
queries = [f"{query_prefix} {query}" for query in queries]
documents = [f"{document_prefix} {document}" for document in documents]

batch_queries = tokenizer(queries, padding=True, truncation=True, return_tensors='pt').to("cuda:0")
batch_documents = tokenizer(documents, padding=True, truncation=True, return_tensors='pt').to("cuda:0")

with torch.no_grad():
    outputs_queries = model(**batch_queries)
    outputs_documents = model(**batch_documents)

# Average Pooling
embeddings_queries = average_pool(outputs_