Model reference · open weights
Muse-Spark-Open is an open-weight language model from meta-models. Muse-Spark-Open-816B-A42B-NVFP4 (NVFP4) weighs 533 GB; the smallest configuration that runs it is 8× A100 80 GB.
For self-hosting, compare this build's requirements below with the GPU you plan to rent.
Summary of the meta-models/Muse-Spark-Open-816B-A42B model card, 2026-10-17. The estimate below is for another build of the family.
What it is
| Released by | meta-models |
|---|---|
| Released | 2026-09-15 |
| Parameters | 822.0B |
| VRAM | 533 GB for the weights |
What it runs on
| Card | Requests at once | Context max | Memory | |
|---|---|---|---|---|
| 8K each | 32K each | |||
| RTX 3060 12 GB … 8× H100 80 GB 13 smaller cards | ||||
| 8× A100 80 GB tensor parallel · FP4 without its speed-up here | 33 | 17 | 842K | 78.2 GB a card |
| 4× B200 180 GB tensor parallel | 32 | 8 | 929K | 176 GB a card |
| 8× H200 141 GB tensor parallel · FP4 without its speed-up here | 206 | 76 | all 1024K | 138 GB a card |
| Requests at once | 8K tokens each | 32K tokens each |
|---|---|---|
| 1 | 539 GB | 541 GB |
| 5 | 546 GB | 554 GB |
| 8 | 551 GB | 564 GB |
| 16 | 565 GB | 591 GB |
| 32 | 593 GB | 645 GB |
| 64 | 649 GB | 752 GB |
One card, with vLLM's small-card settings.
Builds
| Build | Params | Precision | Weights | Smallest setup |
|---|---|---|---|---|
| Muse-Spark-Open-816B-A42B ↗ | 822.0B | BF16 | 1644 GB | none of the reference configurations |
| Muse-Spark-Open-816B-A42B-NVFP4 (above) ↗ | 483.8B | NVFP4 | 533 GB | 8× A100 80 GB |
| Muse-Spark-Open-816B-A42B-GGUF ↗ packaged by unsloth 9 builds: GGUF 796 MB … Q4_K_XL 506 GB |
GGUF | 275 GB | 2× B200 180 GB | |
| Muse-Spark-Open-816B-A42B ↗ packaged by unsloth |
BF16 | |||
| Muse-Spark-Open-816B-A42B-DFlash2 ↗ packaged by nvidia a draft head for speculative decoding that runs beside the full model |
4.4B | BF16 | 8.8 GB | |
| Muse-Spark-Open-816B-A42B-NVFP4 ↗ packaged by unsloth |
483.8B | NVFP4 | 533 GB | 8× A100 80 GB |
From the model card
Authors: Meta Superintelligence Labs Model Release Date: October 2026 License: Muse Model License
Muse Spark Open is an 816B total/42B active parameter open weights model with a context window of 1M tokens. Built for coding and agentic work, Muse Spark Open is natively multimodal. This open weights version has additional safety post-training to make it more robust against misuse as a standalone model without system guardrails. We recommend deploying Muse Spark Open as part of a broader AI system, with additional guardrails suited to the specific use case.
The model can perform well across a wide variety of use cases including:
| Name | Muse Spark Open |
|---|---|
| Model Provider | Meta Platforms, Inc. or Meta Platforms Ireland Limited |
| Date of Release | October 15th, 2026 |
| License | Muse Model License |
| Parameters | 816B total / 42B active |
| Type | Mixture-of-Experts Transformer, with Perception Encoder |
| Context Length | 1M at inference |
| Vocabulary Size | 202,048 |
| Modalities | Input: Text + Image + Video + Audio / Output: Text |
| Languages | Multilingual, trained on data from more than 100 languages |
| Recommended inference parameters | Temperature = 1.0 Top_p = 1.0 Top_k = disabled It is recommended to use Reasoning Effort of high or xhigh for complex problem solving, coding, and agentic tasks. |
| Language Model | Hidden Dimension: 8192 Token Embedding: untied input/output embeddings, vocab × dim = 202,048 × 8192 ≈ ~1.66B Output layer (unembedding layer): vocab × dim = 202,048 × 8192 ≈ ~1.66B Number of Layers: 62 Gated Attention: Yes (output gate and output norm on the attention block; QK-norm enabled) Number of Attention Heads: 128 for Q and 16 for KV (grouped-query attention) Head Dimension: 64 Mixture of Experts: Yes Number of Experts: 256 Experts Activated per Token: 8 (top-k routing) |
| Perception Encoder | Architecture: 1.8B param, 50 layers, 1536 hidden dim, 16 heads, 14x14 patch size, 4096 output dim Learned Absolute + 2D RoPE for positional embedding, LayerNorm + GELU for FFN Sparse attention with factor 4 (every 4th + last layer are full attention) |
Access via API The model is available via API through third party inference providers.
Open Weights on your own system The weights are available for download through Hugging Face. The model is available in BF16 and NVFP4 versions. To run on your own system, the model requires a GPU with at least 700GB of VRAM for NVFP4 and 2TB for BF16. It also requires an inference deployment framework - supported frameworks are SGLang, vLLM, Hugging Face, TensorRT-LLM, Unsloth and Llama.cpp. It is also optimized to run on two Mac Studios with M5 Ultra using MLX.
Please see the developer documentation for more information on how to deploy and use the model.
Inference Parameters You can customize inference parameters to achieve the best performance for your use case.
Training data includes text, images, audio, and video. It includes data drawn from public sources, crawled from the Internet, acquired from third parties, derived from Meta’s products and services, and synthetically generated, curated and enriched by external vendor networks and Meta personnel. Data curation methodologies include cleaning, processing, modifying, deduplication, and filtering, among others.
We compare Muse Spark Open to other leading open-weight models of similar size and architecture. For Muse Spark Open we use xhigh reasoning effort, temperature = 1.0, top_p=1.0, top_k=disabled across benchmarks.
| Benchmark | Muse Spark Open (xhigh) | Inkling (xhigh) | Nemotron 3 Ultra (thinking on) | Beam | Mistral Large 4 | Qwen3.8-2.4T-A95B (xhigh) | GLM 5.3 (max) | Kimi K3 (max) |
|---|---|---|---|---|---|---|---|---|
| General | ||||||||
| Artificial Analysis Intelligence Index | 39 | 26 | 23 | - | 38 | 40 | 45 | 44 |
| Agentic | ||||||||
| JobBench | 61.8 | 31.1 | 27.2 | - | - | 52.7 | 61.4 | 54.3 |
| MCP-Atlas | 89.2 | 76.0 | 63.1 | 78.7 | - | 84.5 | 84.2 | 82.3 |
Quoted from the model card on Hugging Face. The full card is behind the Hugging Face link above.
How it works