MiniCPM5-Midtrain

English |

Highlights

We are releasing MiniCPM5-2B, the second model in the MiniCPM5 series, following MiniCPM5-1B. It is a dense 2B Transformer that scales up the same training recipe, built for on-device, local deployment, and resource-constrained scenarios, reaching 2B-class open-source SOTA.

🏆 2B-class open-source SOTA: compared with strong open-source models of similar size, MiniCPM5-2B achieves SOTA performance within this comparison set. It remains competitive with 4B-class models overall, while showing its advantages over models of comparable size in coding, mathematics, long-context understanding, tool use, and agentic tasks.

#capability-comparison-radar {
  --foreground: #171717;
  display: block; width: 100%; max-width: 620px; margin: 0 auto; background: #fff;
  font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei", sans-serif;
}
#capability-comparison-radar svg { display: block; width: 100%; height: auto; background: #fff; }
#capability-comparison-radar .title { fill: var(--foreground); font-size: 16px; font-weight: 600; letter-spacing: 0; }
#capability-comparison-radar .axis-label { fill: #333; font-size: 12px; font-weight: 600; }
#capability-comparison-radar .ring { fill: none; stroke: rgba(128,128,128,.18); stroke-width: .8; }
#capability-comparison-radar .spoke { stroke: rgba(128,128,128,.25); stroke-width: .8; }
#capability-comparison-radar .ring-label { fill: #8a8a8a; font-size: 9px; }
#capability-comparison-radar .series { stroke-linejoin: round; }
#capability-comparison-radar .series.primary { stroke-width: 2.4; }
#capability-comparison-radar .legend-label { fill: #171717; font-size: 12px; font-weight: 600; }
#capability-comparison-radar .legend-average { fill: #666; font-size: 11px; }
#capability-comparison-radar .legend-swatch { rx: 3; }
@media (max-width: 900px) { #capability-comparison-radar { overflow-x: auto; } #capability-comparison-radar svg { min-width: 620px; } }

📂 Open High-Quality Data: Alongside the model, we are releasing the high-quality training datasets behind it as part of the UltraData family: UltraX, a high-quality web pre-training dataset; UltraData-Code, featuring L0–L3 tiered code data management to drive a significant leap in coding capabilities; UltraData-SFT-Agent-2609, comprising 500K agent training samples to enhance comprehensive on-device agent capabilities; and UltraData-RL-2609, with 80K+ high-quality RL training samples covering mathematics, code, general knowledge, and long-context reasoning.

Model List

Use this directory to choose the model format that matches your runtime:

MiniCPM5-2B

• MiniCPM5-2B · ModelScope · BF16 final release (post-trained with RL + OPD)
• MiniCPM5-2B-SFT · ModelScope · BF16 SFT-only checkpoint (before RL / OPD)
• MiniCPM5-2B-Midtrain · ModelScope · BF16 mid-training checkpoint (before SFT) 👈 you are here
• MiniCPM5-2B-Base · ModelScope · BF16 base checkpoint (pre-training only)
• MiniCPM5-2B-GGUF · ModelScope · GGUF for llama.cpp / Ollama / LM Studio
• MiniCPM5-2B-MLX · ModelScope · MLX / 4bit for Apple Silicon
• MiniCPM5-2B-GPTQ · ModelScope · GPTQ / 4bit quantized model
• MiniCPM5-2B-DSpark · ModelScope · DSpark draft model for inference acceleration

MiniCPM5-1B

• MiniCPM5-1B · ModelScope · BF16 final release (post-trained with RL + OPD)
• MiniCPM5-1B-SFT · ModelScope · BF16 SFT-only checkpoint (before RL / OPD)
• MiniCPM5-1B-Base · ModelScope · BF16 base checkpoint (pre-training only)
• MiniCPM5-1B-GGUF · ModelScope · GGUF for llama.cpp / Ollama / LM Studio
• MiniCPM5-1B-MLX · ModelScope · MLX / 4bit for Apple Silicon

Model Information

MiniCPM5-2B has the following features:

• Type: Causal Language Model
• Architecture: Standard LlamaForCausalLM
• Number of Parameters: 2,516,756,480
• Number of Non-Embedding Parameters: 1,981,982,720
• Number of Layers: 42
• Number of Attention Heads (GQA): 16 for Q and 2 for KV
• Context Length: 131,072

Introduction

MiniCPM5-2B is the second model in the MiniCPM5 series. It i