NVIDIA-Nemotron-3.5-Bolt

Description

NVIDIA-Nemotron-3.5-Bolt-E4B-BF16 is a small language model (SLM) further post-trained from Gemma-4-E4B-it by NVIDIA. This is the full-precision (BF16) release. Nemotron-3.5-Bolt-E4B carries forward Gemma’s strong conversational capabilities and further improves multi-turn tool use, instruction following, and input grounding for fast local voice/device assistants, interactive personas, and embedded/edge interfaces. Designed for low-memory, low-TTFT operation, the model allows agents to respond quickly while preserving headroom for applications or adjacent model pipelines.

For speculative decoding, use the companion DFlash drafter. For the Q4_K_M quantized version, see NVIDIA-Nemotron-3.5-Bolt-E4B-4GB.

This model is ready for commercial or non‑commercial use.

Model Summary

Model Overview

Model Developer: NVIDIA Corporation

Model Dates: December 2025 - May 2026

Data Freshness:

• The post-training data has a cutoff date of May 2026.

What is Nemotron?

NVIDIA Nemotron™ is a family of open models with open weights, training data, and recipes, delivering leading efficiency and accuracy for building specialized AI agents.

Quick Start

To get quickly started on a single H100 you can use the following command.

Run it with vLLM! (vLLM v0.31.0: vllm/vllm-openai:v0.31.0)

Without Speculative Decoding

export MODEL=nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16

vllm serve "$MODEL" \
  --served-model-name "nemotron-bolt" \
  --host 0.0.0.0 \
  --port 8000 \
  --dtype bfloat16 \
  --tensor-parallel-size 1 \
  --gpu-memory-utilization 0.85 \
  --kv-cache-dtype auto \
  --max-model-len 65536 \
  --max-num-seqs 64 \
  --max-num-batched-tokens 65536 \
  --chat-template "$MODEL/chat_template.jinja" \
  --enable-auto-tool-choice \
  --tool-call-parser gemma4 \
  --reasoning-parser gemma4 \
  --limit-mm-per-prompt '{"image":0,"video":0,"audio":0}'

With DFlash Speculative Decoding

Use the same downloaded BF16 checkpoint with its companion DFlash drafter. Run one serving configuration at a time. This example uses a single concurrent sequence and seven speculative tokens; tune these settings for your workload.

export MODEL=nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16
export DRAFTER_CKPT=nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16-DFlash-NVFP4

vllm serve "$MODEL" \
  --served-model-name "nemotron-bolt" \
  --host 0.0.0.0 \
  --port 8000 \
  --dtype bfloat16 \
  --tensor-parallel-size 1 \
  --gpu-memory-utilization 0.85 \
  --kv-cache-dtype auto \
  --max-model-len 65536 \
  --max-num-seqs 1 \
  --max-num-batched-tokens 65536 \
  --chat-template "$MODEL/chat_template.jinja" \
  --enable-auto-tool-choice \
  --tool-call-parser gemma4 \
  --reasoning-parser gemma4 \
  --limit-mm-per-prompt '{"image":0,"video":0,"audio":0}' \
  --speculative-config "{
    \"method\": \"dflash\",
    \"model\": \"$DRAFTER_CKPT\",
    \"num_speculative_tokens\": 7
  }"

Sample Request

The same request works with either serving configuration.

curl -s "http://:8000/v1/chat/completions" \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "nemotron-bolt",
    "messages": [
      {"role": "user", "content": "What is 17 multiplied by 23?"}
    ],
    "temperature": 0.0,
    "max_tokens": 512,
    "skip_special_tokens": false,
    "chat_template_kwargs": {
      "enable_thinking": true
    }
  }'

For more details on how to deploy and use the model — see the Quick Start Guide below!

License/Terms of Use

Governing Terms: Use of this model is governed by the OpenMDW License Agreement, version 1.1. This model has been post-trained on the Google Gemma 4 model, and use of the Google Gemma 4 model is governed by Apache License, Version 2.0

Benchmarks

We evaluated our model on the following benchmarks: