NVIDIA-Nemotron-3.5-Bolt-4GB

Description

NVIDIA-Nemotron-3.5-Bolt-E4B-4GB is a small language model (SLM) further post-trained from Gemma-4-E4B-it by NVIDIA. Nemotron-3.5-Bolt-E4B carries forward Gemma’s strong conversational capabilities and further improves multi-turn tool use, instruction following, and input grounding for fast local voice/device assistants, interactive personas, and embedded/edge interfaces. Designed for low-memory, low-TTFT operation, the model allows agents to respond quickly while preserving headroom for applications or adjacent model pipelines.

For speculative decoding, use the companion DFlash drafter. For the full-precision (BF16) version, see NVIDIA-Nemotron-3.5-Bolt-E4B-BF16.

This model is ready for commercial or non‑commercial use.

Model Summary

Model Overview

Model Developer: NVIDIA Corporation

Model Dates: December 2025 - May 2026

Data Freshness:

• The post-training data has a cutoff date of May 2026.

What is Nemotron?

NVIDIA Nemotron™ is a family of open models with open weights, training data, and recipes, delivering leading efficiency and accuracy for building specialized AI agents.

Quick Start

export MODEL_REPO=nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB
export MODEL_FILE=Nemotron-3.5-Bolt-E4B-Q4_K_M.gguf

export DRAFTER_REPO=nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB-DFlash
export DRAFTER_FILE=Nemotron-3.5-Bolt-E4B-DFlash-BF16.gguf

export MODEL_DIR="$PWD/bolt-models"

hf download "$MODEL_REPO" "$MODEL_FILE" \
  --local-dir "$MODEL_DIR"

hf download "$DRAFTER_REPO" "$DRAFTER_FILE" \
  --local-dir "$MODEL_DIR"

Without drafter

docker run --rm --gpus all \
  -p 8000:8000 \
  -v "$MODEL_DIR:/models:ro" \
  ghcr.io/ggml-org/llama.cpp:server-cuda13-b11459 \
  -m "/models/$MODEL_FILE" \
  --alias nemotron-bolt \
  --host 0.0.0.0 \
  --port 8000 \
  -c 65536 \
  -ngl 999 \
  --jinja

With DFlash drafter

docker run --rm --gpus all \
  -p 8000:8000 \
  -v "$MODEL_DIR:/models:ro" \
  ghcr.io/ggml-org/llama.cpp:server-cuda13-b11459 \
  -m "/models/$MODEL_FILE" \
  --spec-draft-model "/models/$DRAFTER_FILE" \
  --spec-type draft-dflash \
  --spec-draft-n-max 7 \
  --spec-draft-ngl 999 \
  --alias nemotron-bolt \
  --host 0.0.0.0 \
  --port 8000 \
  -c 65536 \
  -ngl 999 \
  --jinja

curl --fail-with-body -sS \
  http://localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "nemotron-bolt",
    "messages": [
      {"role": "user", "content": "What is 17 multiplied by 23?"}
    ],
    "temperature": 0.0,
    "max_tokens": 512,
    "chat_template_kwargs": {
      "enable_thinking": true
    }
  }'

For more details on how to deploy and use the model — see the Quick Start Guide below!

License/Terms of Use

Governing Terms: Use of this model is governed by the OpenMDW License Agreement, version 1.1. This model has been post-trained on the Google Gemma 4 model, and use of the Google Gemma 4 model is governed by Apache License, Version 2.0

Benchmarks

Reasoning Benchmark Evaluations

Benchmarks

We evaluated our model on the following benchmarks: