NVIDIA-Nemotron-3.5-Bolt-24GB

Description

NVIDIA Nemotron 3.5 Bolt 74B-A9B 24GB is a vision-language model (VLM) designed to run everyday agentic tasks locally, using a native 2-bit quantization to fit in 24GB of memory and run alongside other applications. It optimizes intelligence and speed per GB. It aims to achieve the optimal balance between speed and intelligence, being faster than most popular dense models while smarter than most popular MoE models with a similar memory footprint.

This model is ready for commercial or non‑commercial use.

Model Summary

Model Overview

Model Developer: NVIDIA Corporation

Model Dates: December 2025 - May 2026

Data Freshness:

• The post-training data has a cutoff date of May 2026.

What is Nemotron?

NVIDIA Nemotron™ is a family of open models with open weights, training data, and recipes, delivering leading efficiency and accuracy for building specialized AI agents.

Quick Start

export MODEL_DIR="$PWD/models/nemotron-bolt"

hf download nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF \
  NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB.gguf \
  mmproj-NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB.gguf \
  --local-dir "$MODEL_DIR"

hf download nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-LiLiCorr-GGUF \
  NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-LiLiCorr.gguf \
  --local-dir "$MODEL_DIR"

export MODEL="$MODEL_DIR/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB.gguf"
export MMPROJ="$MODEL_DIR/mmproj-NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB.gguf"
export SPEC_DRAFT="$MODEL_DIR/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-LiLiCorr.gguf"

./build/bin/llama-server \
  --model "$MODEL" \
  --mmproj "$MMPROJ" \
  --batch-size 2048 \
  --ubatch-size 512 \
  --ctx-size 98304 \
  --parallel 1 \
  -ngl 999 \
  --spec-type draft-dflash \
  --spec-draft-n-max 6 \
  --spec-draft-model "$SPEC_DRAFT" \
  --spec-draft-ngl 999 \
  --cache-type-k q8_0 \
  --cache-type-v q8_0 \
  --temp 1.0 \
  --top-p 0.95 \
  --flash-attn on \
  --jinja

For more details on how to deploy and use the model — see the Quick Start Guide below!

License/Terms of Use

Governing Terms: Use of the models is governed by the OpenMDW License Agreement, version 1.1.

Benchmarks

*The reported MCP-Atlas score for Qwen3.6-35B-A3B-BF16 is 62.8; we measured 51.2 on the 500 public MCP-Atlas tasks.

Deployment Geography: Global

Use Case

Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF is optimized for use in local agent harnesses and general agentic tasks involving tool calling, web browsing, and terminal use. English and coding languages are the primary languages, with Spanish, French, German, Italian, and Japanese also being supported.

Release Date

Hugging Face 10/15/2026 via https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF

Reference(s)

• NVIDIA Nemotron 3 model family on Hugging Face
• NVIDIA Nemotron 3.5 Bolt 74B A9B 24GB Technical Report
• NVIDIA Nemotron 3.5 Bolt 74B A9B 24GB Technical Blog

Model Architecture

• Architecture Type: Mamba2-Transformer Hybrid Mixture of Experts (MoE) with vision encoder
• Network Architecture: Nemotron Hybrid LatentMoE
  • Nemotron 3.5 Super LLM — hybrid Mamba-2 + MoE + Attention backbone
  • C-RADIO v4-H — vision encoder for image and video frames, connected to the backbone through an MLP projector
• Number of model parameters: 74B Total / 9B Active

Model Design

The model was compressed from Nemotron 3.5 Super VL 120B-A12B. The model supports up to 256K tokens of context. The prompt, visual inputs, conversation history, tool results, and generated response share this context window. Individual deployment recipes may use a smaller context window to accommodate the available GPU memory and concurrency.

Train