Model reference · open weights
NVLM-D is an open-weight language model from NVIDIA. NVLM-D-72B (BF16) weighs 177 GB; the smallest configuration that runs it is 2× H200 141 GB.
For self-hosting, compare this build's requirements below with the GPU you plan to rent.
Summary of the nvidia/NVLM-D-72B model card, 2026-10-01
What it is
| Released by | NVIDIA |
|---|---|
| Released | 2024-09-30 |
| Parameters | 79.4B |
| VRAM | 177 GB for the weights |
What it runs on
| Card | Requests at once | Context max | Memory | |
|---|---|---|---|---|
| 8K each | 32K each | |||
| RTX 3060 12 GB … B200 180 GB 12 smaller cards | ||||
| 2× H200 141 GB tensor parallel | 31 | 7 | all 32K | 138 GB a card |
| 4× H100 80 GB tensor parallel | 39 | 9 | all 32K | 78.1 GB a card |
| 4× A100 80 GB tensor parallel | 47 | 11 | all 32K | 78.2 GB a card |
| 2× B200 180 GB tensor parallel | 58 | 14 | all 32K | 176 GB a card |
| 4× RTX PRO 6000 Blackwell 96 GB tensor parallel | 63 | 15 | all 32K | 93.8 GB a card |
| Requests at once | 8K tokens each | 32K tokens each |
|---|---|---|
| 1 | 182 GB | 190 GB |
| 5 | 193 GB | 233 GB |
| 8 | 201 GB | 265 GB |
| 16 | 222 GB | 351 GB |
| 32 | 265 GB | 523 GB |
| 64 | 351 GB | 866 GB |
One card, with vLLM's small-card settings.
From the model card
This family of models performs vision-language and text-only tasks including optical character recognition, multimodal reasoning, localization, common sense reasoning, world knowledge utilization, and coding.
This model is ready for non-commercial use.
Governing Terms: Deed - Attribution-NonCommercial 4.0 International - Creative Commons.
Additional Information: LICENSE · Qwen/Qwen2-72B-Instruct at main for Qwen2-72B-Instruct and The MIT License – Open Source Initiative for InternViT-6B-448px-V1-2.
Today (September 17th, 2024), we introduce NVLM 1.0, a family of frontier-class multimodal large language models (LLMs) that achieve state-of-the-art results on vision-language tasks, rivaling the leading proprietary models (e.g., GPT-4o) and open-access models (e.g., Llama 3-V 405B and InternVL 2). Remarkably, NVLM 1.0 shows improved text-only performance over its LLM backbone after multimodal training.
In this repo, we are open-sourcing NVLM-1.0-D-72B (decoder-only architecture), the decoder-only model weights and code for the community.
Paper Inference Code (HF) Training Code Website
We train our model with legacy Megatron-LM and adapt the codebase to Huggingface for model hosting, reproducibility, and inference. We observe numerical differences between the Megatron and Huggingface codebases, which are within the expected range of variation. We provide the results from both the Huggingface codebase and the Megatron codebase for reproducibility and comparison with other models.
Results (as of September 17th, 2024) in the multimodal benchmarks are as follows:
| Benchmark | MMMU (val / test) | MathVista | OCRBench | AI2D | ChartQA | DocVQA | TextVQA | RealWorldQA | VQAv2 |
|---|---|---|---|---|---|---|---|---|---|
| NVLM-D 1.0 72B (Huggingface) | 58.7 / 54.9 | 65.2 | 852 | 94.2 | 86.0 | 92.6 | 82.6 | 69.5 | 85.4 |
| NVLM-D 1.0 72B (Megatron) | 59.7 / 54.6 | 65.2 | 853 | 94.2 | 86.0 | 92.6 | 82.1 | 69.7 | 85.4 |
| Llama 3.2 90B | 60.3 / - | 57.3 | - | 92.3 | 85.5 | 90.1 | - | - | 78.1 |
| Llama 3-V 70B | 60.6 / - | - | - | 93.0 | 83.2 | 92.2 | 83.4 | - | 79.1 |
| Llama 3-V 405B | 64.5 / - | - | - | 94.1 | 85.8 | 92.6 | 84.8 | - | 80.2 |
| InternVL2-Llama3-76B | 55.2 / - | 65.5 | 839 | 94.8 | 88.4 | 94.1 | 84.4 | 72.2 | - |
| GPT-4V | 56.8 / 55.7 | 49.9 | 645 | 78.2 | 78.5 | 88.4 | 78.0 | 61.4 | 77.2 |
| GPT-4o | 69.1 / - | 63.8 | 736 | 94.2 | 85.7 | 92.8 | - | - | - |
| Claude 3.5 Sonnet | 68.3 / - | 67.7 | 788 | 94.7 | 90.8 | 95.2 | - | - | - |
| Gemini 1.5 Pro (Aug 2024) | 62.2 / - | 63.9 | 754 | 94.4 | 87.2 | 93.1 | 78.7 | 70.4 | 80.2 |
| Tasks | Backbone LLM | MMLU | GSM8K | MATH | HumanEval | Avg. Accuracy |
|---|---|---|---|---|---|---|
| Proprietary | ||||||
| GPT-4.0 | N/A | 88.7 | - | 76.6 | 90.2 | - |
| Gemini Pro 1.5 (Aug 2024) | N/A | 85.9 | 90.8 | 67.7 | 84.1 | 82.1 |
| Claude 3.5 Sonnet | N/A | 88.7 | 96.4 | 71.1 | 92.0 | 87.0 |
| Open LLM | ||||||
| (a) Nous-Hermes-2-Yi-34B | N/A | 75.5 | 78.6 | 21.8 | 43.3 | 54.8 |
| (b) Qwen-72B-Instruct | N/A | 82.3 | 91.1 | 59.7 | 86.0 | 79.8 |
| (c) Llama-3-70B-Instruct | N/A | 82.0 | 93.0 | 51.0 | 81.7 | 76.6 |
| (d) Llama-3.1-70B-Instruct | N/A | 83.6 | 95.1 | 68.0 | 80.5 | 81.8 |
| (e) Llama-3.1-405B-Instruct | N/A | 87.3 | 96.8 | 73.8 | 89.0 | 86.7 |
| Open Multimodal LLM | ||||||
| VILA-1.5 40B | (a) | 73.3 | 67.5 | 16.8 | 34.1 | 🥶 47.9 (-6.9) |
| LLaVA-OneVision 72B | (b) | 80.6 | 89.9 | 49.2 | 74.4 | 🥶 73.5 (-6.3) |
| InternVL-2-Llama3-76B | (c) | 78.5 | 87.1 | 42.5 | 71.3 | 🥶 69.9 (-6.7) |
| *Llama 3-V 70B | (d) | 83.6 | 95.1 | 68.0 | 80.5 | 🙂 81.8 (0) |
| *Llama 3-V 405B | (e) | 87.3 | 96.8 | 73.8 | 89.0 | 🙂 86.7 (0) |
| NVLM-D 1.0 72B (Megatron) | (b) | 82.0 | 92.9 | 73.1 | 88.4 | 🥳 84.1 (+4.3) |
| NVLM-D 1.0 72B (Huggingface) | (b) | 81.7 | 93.2 | 73.1 | 89.0 | 🥳 84.3 (+4.5) |
Network Architecture: Decoder-Only Transformer
Text-only LLM backbone: [Qwen2-72B-Instruct](htt
Quoted from the model card on Hugging Face. The full card is behind the Hugging Face link above.