Step-5

We are excited to release Step-5-Preview, our flagship foundation model for real-world agentic work. It is a 600B-parameter sparse Mixture-of-Experts model with 27B active parameters, a 1M-token context window, and native support for text, image, and video inputs. Try it via our API, or deploy locally with vLLM / SGLang.

📖 Table of Contents

• Introduction
• Key Features
• Model Architecture
• Model Specifications
• Training Data
• Benchmark Results
• Agentic Capabilities
• Real-World Use Cases
• Quickstart
• Deployment
• Evaluation
• Limitations
• Ethical Considerations
• Hardware Requirements
• Performance Metrics
• Citation
• License
• Contact

🚀 Introduction

Step-5-Preview is StepFun's flagship foundation model, designed from the ground up for real-world agentic tasks. It targets professional domains such as AI coding, software engineering, professional knowledge work, and financial analysis.

StepFun's core philosophy for Step 5 is the "Pareto Frontier" — achieving the optimal balance between intelligence and cost. While previous scaling efforts focused on trading more compute for stronger intelligence, the next phase requires improving the efficiency of converting compute into intelligence.

• 600B total parameters, only 27B active — near-frontier performance at a fraction of the compute. • 1M-token context window without proportional cost increases. • Competitive benchmark scores against models with 3–5× more parameters. • Built for agents — long-horizon reasoning, tool use, and autonomous execution.

Step-5-Preview represents a generational leap, with StepFun skipping the entire Step 4.x line entirely, going directly from Step-3.7-Flash to Step 5. This decision reflects the magnitude of improvement achieved in this release.

✨ Key Features

• Sparse Mixture-of-Experts (MoE): 600B total parameters, 27B active per token (~4.5% sparsity).
• 1M-Token Context Window: Equivalent to ~1,500 A4 pages, enabled by Sparse GQA.
• Multimodal Input: Text, image, and video (MP4, QuickTime, Matroska; ≤128 MB; ≤5 min recommended).
• Configurable Reasoning Effort: low, medium, high / xhigh.
• Parallel Tool Calling: Natively supported for agentic workflows.
• Strict JSON Schema Output: Reliable integration into structured systems.
• OpenAI-Compatible API: Available via Step API and third-party gateways.
• Open Weights: BF16 checkpoint available now under TypeSafeAI/Step-5-Preview-BF16.

🏗️ Model Architecture

92-Layer "Narrow but Deep" Design

Step-5-Preview uses a 92-layer Transformer with a narrow-deep configuration. This design is specifically intended to create longer information propagation paths for implicit multi-hop reasoning during long prefill operations.

Sparse Grouped-Query Attention (GQA) with Block-Wise Token Merging

To handle the 1M-token context window efficiently, Step-5-Preview introduces Sparse GQA with block-wise token merging. This mechanism uses sparse indexing to select only historical information relevant to the current task, reducing the number of tokens that actually enter attention computation. StepFun states this cuts indexer and top-k selection costs to approximately one-eighth of a denser baseline.

Step 5 Preview achieves near-frontier performance with 600B total parameters but only

Multimodal Encoder

The model incorporates a unified multimodal encoder that processes text, images, and video frames into a shared latent space. Video is sampled at adaptive frame rates and encoded with temporal attention, allowing the model to understand motion and long-range dependencies in screen recordings, demonstrations, and real-world footage.

📋 Model Specifications

📚 Training Data

Step-5-Preview was trained on a massive, carefully curated corpus spanning:

• Code repositories from multiple languages (Python, C++, Rust, JavaScript, Go, etc.)
• Technical documentation, API references, and software engineering forums
• Scientific papers in computer science, mathematics, physics, and finance
• Financial reports, earnings calls, and market analyses
• Multimodal data including screenshots, UI mockups, video tutorials, and screen recordings
• Agentic trajectories from simulated and real tool-use environments

The data mixture was optimized for long-horizon reasoning and tool use, with a strong emphasis on real-world professional tasks. All data was filtered for quality, safety, and li