NVIDIA-Nemotron-3.5-Bolt-24GB-LiLiCorr

Description

NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-LiLiCorr-GGUF is the LiLiCorr speculative-decoding checkpoint for NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF. The compatible target is a hybrid LatentMoE vision-language model for reasoning, chat, and agent workflows. For more information, see the NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF target card.

This checkpoint is not a standalone language model. It proposes draft token candidates that the compatible 24GB target verifies; the target produces the user-visible text.

This model is ready for commercial or non-commercial use.

License/Terms of Use

GOVERNING DOWNLOAD TERMS: Use of this model is governed by the OpenMDW-1.1 model license.

Deployment Geography

Global

Use Case

Developers can pair this draft with Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF for text generation. The target supports reasoning, chat, retrieval-augmented generation, visual understanding, and agent workflows. The draft accelerates generation through speculative decoding.

Release Date

Hugging Face 10/15/2026 via https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-LiLiCorr-GGUF

References

• NVIDIA Model Optimizer
• NVIDIA Nemotron-3.5-Bolt-74B-A9B-BF16 reasoning model card
• NVIDIA Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF reasoning model card
• OpenMDW License Agreement, version 1.1
• LiLiCorr: Lightweight Likelihood Correlation of Parallel Drafts for Speculative Decoding

Model Architecture

Architecture Type: Dense GQA (Dense MLP + GQA Attention)

Network Architecture: Five dense feed-forward layers use grouped-query attention (GQA). Draft attention is bidirectional within the draft block. All five layers use a 1024-token per-head sliding window and attention sinks, with a grouped dynamic convolution around each attention and MLP sublayer. A separate two-layer LiLiCorr head correlates the drafter's top-8 candidates at each draft slot.

Number of Model Parameters: 1.113B draft parameters, excluding shared target embeddings and the LM head. The two-layer LiLiCorr head accounts for 37M of them.

Target Model Architecture: Hybrid LatentMoE vision-language model with 74B total parameters and 9B active parameters.

Input

The inputs below describe the target-model serving interface. LiLiCorr drafts tokens using representations from the target model.

Input Type(s): Text Input Format(s): Text: String Input Parameters: Text: One-Dimensional (1D) Other Properties Related to Input: The target model supports up to 256K tokens of context. Supported languages include English, French, Spanish, Italian, German, Japanese, Hindi, Korean, Brazilian Portuguese, and Chinese.

Output

• Output Type(s): Text
• Output Format: String
• Output Parameters: One-Dimensional (1D): Sequences
• Other Properties Related to Output: Maximum context length up to 256K tokens. Supports JSON output format, reasoning output with chain-of-thought, and tool calling.

Other Properties Related to Output: Outputs can include responses, reasoning traces, tool-use content, and structured data. The chat template and application configuration determine the output form.

Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA’s hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.

Software Integration

Supported Runtime Engine(s):

• llama.cpp

Supported Hardware Microarchitecture Compatibility:

• NVIDIA Blackwell, including DGX Spark (GB10)

Preferred Operating System(s):

• Linux
• Windows

The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels are essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment.

Model Version(s)

The model is a NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF speculative decoding version quantized with Model Optimizer @ b5f10f8

Training and Evaluation Datasets

Training Dataset

For target training details, see the NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF target card.

Link: Nemotron-Post-Training-Dataset-v2, Nemotron-RL-Super-Training-Blends, Nemotron-Post-Training-v3, Nemotron-Instruction-Following-Chat-v1, and Nemotron-Science-v1. Only dataset prompts and conversation context were used for data synthesis. The original responses were not training targets. Every training response was generated by Nemotron-3.5-Bolt at temperature 1.0. The draft was first trained on responses from a Nemotron-3.5-Bolt post-training checkpoint, then fine-tuned on responses from the released checkpoint.

Data Modality: Text

Training Data Size: 1 Billion to 10 Trillion Tokens