NVIDIA-Nemotron-3.5-Bolt-4GB-DFlash

Description:

The NVIDIA-Nemotron-3.5-Bolt-E4B-4GB-DFlash model is the GGUF (a binary model file format used by llama.cpp) version of NVIDIA-Nemotron-3.5-Bolt-E4B-BF16-DFlash, the DFlash speculative decoding draft checkpoint for NVIDIA-Nemotron-3.5-Bolt-E4B, a reasoning language model post-trained by NVIDIA from Google's Gemma 4 E4B IT. It is converted to GGUF in BF16 (bfloat16) for use with llama.cpp and must be loaded together with the GGUF version of the target model, Nemotron-3.5-Bolt-E4B-4GB. The draft model does not include token embeddings or an output head, and uses those of the target model at runtime. The model is intended for lower-latency local serving of NVIDIA-Nemotron-3.5-Bolt-E4B on workstation GPUs.

This model is ready for commercial or non-commercial use.

License/Terms of Use:

GOVERNING DOWNLOAD TERMS: Use of the model is governed by the OpenMDW License Agreement, version 1.1. ADDITIONAL INFORMATION: Apache License 2.0 (Gemma 4 E4B IT).

Deployment Geography:

Global

Use Case:

Developers running NVIDIA-Nemotron-3.5-Bolt-E4B locally with llama.cpp for reasoning, chat, tool use, and agentic workflows that benefit from lower-latency speculative decoding on workstation GPUs. This release is intended for DFlash-assisted serving together with NVIDIA-Nemotron-3.5-Bolt-E4B-GGUF rather than as a standalone model checkpoint.

Release Date:

Hugging Face 10/23/2026 via https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB-DFlash

References

• NVIDIA Model Optimizer: https://github.com/NVIDIA/Model-Optimizer
• llama.cpp: https://github.com/ggml-org/llama.cpp
• NVIDIA-Nemotron-3.5-Bolt-E4B-DFlash model card
• Nemotron-3.5-Bolt-E4B-BF16 model card
• Nemotron-3.5-Bolt-E4B-4GB model card
• Gemma 4 E4B IT model card
• OpenMDW License Agreement, version 1.1
• DFlash: Block Diffusion for Flash Speculative Decoding

Model Architecture:

Architecture Type: Transformer Network Architecture: Other - DFlash draft model (five Qwen3-style decoder layers) Number of Model Parameters: 360M (360,480,000), stored in BF16 in a single GGUF file of 736,812,096 bytes (736.81 MB / 702.68 MiB). The draft model shares the token embeddings and output head of the target model, so they are not included in this checkpoint.

For more information about the underlying model's architecture, please see Nemotron-3.5-Bolt-E4B-BF16.

Input:

Input Type(s): Text Input Format(s): String Input Parameters: One-Dimensional (1D) Other Properties Related to Input: Maximum context length of 131,072 tokens, set in llama.cpp with -c. The draft model attends to the most recent 1,024 tokens through its sliding-window attention.

Output:

Output Type(s): Text Output Format: String Output Parameters: One-Dimensional (1D): Sequences Other Properties Related to Output: The draft model proposes up to 7 candidate tokens per decoding step, and the target model verifies them, so the final output is generated by the target model.

Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA's hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.

Software Integration:

Supported Runtime Engine(s):

• llama.cpp b11808

Supported Hardware Microarchitecture Compatibility:

• NVIDIA RTX 5090

Preferred Operating System(s):

• Linux

The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels are essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment.

Model Version(s):

v1.0: NVIDIA-Nemotron-3.5-Bolt-E4B-DFlash-GGUF is the BF16 GGUF version of NVIDIA-Nemotron-3.5-Bolt-E4B-DFlash, which was trained with NVIDIA Model Optimizer 0.48.0. It was converted with llama.cpp convert_hf_to_gguf.py at commit [4912ac2]. To integrate it into an AI system, load it with -md alongside NVIDIA-Nemotron-3.5-Bolt-E4B-GGUF in llama-server (see Usage); applications then query the llama-server endpoint as usual.

Training and Evaluation Datasets:

Training Dataset:

For more information about the underlying NVIDIA-Nemotron-3.5-Bolt-E4B model, please visit the model card - Nemotron-3.5-Bolt-E4B-BF16.

Link: Nemotron-Post-Training-Dataset-v2, only prompts from the datasets were used for data synthesis (the original responses from GPT were not used), which is then used to train the DFlash modules. Data Modality: Text Text Training Data Size: 1 Billion to 10 Trillion Tokens Data Collection Method by dataset: Hybrid: Automated, Synthetic Labeling Method by dataset: Hybrid: Automated, Synthetic Properties: About 3.7