Model reference · open weights
Kimi-K3 is an open-weight embedding model from moonshotai. Kimi-K3-NVFP4 (NVFP4) weighs 1646 GB; it needs more than the reference configurations below.
For self-hosting, compare this build's requirements below with the GPU you plan to rent.
Summary of the RedHatAI/Kimi-K3-NVFP4 model card, 2026-10-01
What it is
| Released by | moonshotai |
|---|---|
| Released | 2026-07-30 |
| Parameters | 1588.7B |
| VRAM | 1646 GB for the weights |
What it runs on
| Card | Runs | Memory |
|---|---|---|
| RTX 3060 12 GB … B200 180 GB | does not fit |
From the model card
This is a quantized version of moonshotai/Kimi-K3 with MoE layers quantized to NVFP4 for accelerated inference.
This model is intended for deployment with vLLM. You can serve the model using
vllm serve RedHatAI/Kimi-K3-NVFP4 \
--tensor-parallel-size 8 \
--trust_remote_code \
--load-format instanttensor \
--reasoning-parser kimi_k3 \
--language-model-only # optional
May require https://github.com/vllm-project/vllm/pull/50500 to run
This model was compressed using LLM Compressor. For more information, please contact Kyle Sayers via the vLLM Developers Slack or ksayers@redhat.com.
| Benchmark | moonshotai/Kimi-K3 | RedHatAI/Kimi-K3-NVFP4 |
|---|---|---|
| GPQA | 93.5 | 91.0 |
inspect eval hf/Idavidrein/gpqa/diamond \
--model vllm/RedHatAI/Kimi-K3-NVFP4 \
--reasoning-effort high \
--model-base-url http://localhost:8000/v1
Quoted from the model card on Hugging Face. The full card is behind the Hugging Face link above.