Kandinsky-WM-1.0-I2V-5s-PH

Image-to-Video generation for Physical AI: autonomous driving · robotics · general physics

🤗 Checkpoints collection · 🧩 Base model (Kandinsky 5.0) · 💻 GitHub · 📝 Habr article

This repo holds the 🌍 General Physics checkpoint of the Kandinsky WM 1.0 family — DiT soar, published here in two layouts: a diffusers pipeline (model_index.json, transformer/, vae/, text_encoder*/) and the GitHub-code DiT at model/soar.safetensors (same weights). The other two variants are in the Kandinsky WM 1.0 collection.

Contents

• Overview
• Model Zoo
• Examples — 🚗 Autonomous Driving · 🤖 Robotics · 🌍 General Physics
• Results — PAI-Bench-G · RBench · Physics-IQ Verified
• Quickstart — Path A: GitHub code · Path B: Diffusers
• Acknowledgements

Overview

Kandinsky WM (World Model) 1.0 is a family of Image-to-Video models that adapt Kandinsky 5.0 Video Lite — a 2B-parameter latent video diffusion model (a Kandinsky5Transformer3DModel DiT paired with a HunyuanVideo VAE and Qwen2.5-VL + CLIP text encoders, trained with flow matching) — to Physical AI: video generation that is not only visually convincing but physically plausible — consistent scene geometry, object dynamics, interactions and cause-and-effect — so the model can serve as a source of synthetic training data and as a building block for world models and simulators.

The base model was domain-adapted on large corpora of 5-second scenes (first text-to-video, then a mixed text/image-to-video regime that preserves first-frame continuation), spanning autonomous driving, robotics, and a general domain (industrial processes, physical phenomena, and human–object / human–human interaction). Each checkpoint is then reinforcement-learning post-trained with GRPO against a reward model that scores the physical plausibility of the generated scene, steering the generator toward more faithful geometry, object behaviour and interactions than domain fine-tuning alone.

Each checkpoint generates 5-second, 121-frame clips at 768×512.

Model Zoo

Three domain checkpoints, same architecture — only the DiT weights differ (the VAE / text encoders are byte-identical across all three). Each is published as a one-line diffusers pipeline; a GitHub-code (DiT-only) copy lives in the same Hub repo used purely as file storage.

Examples

Four Kandinsky WM 1.0 image-to-video generations per domain (top-ranked by an internal visual review). Each grid shows the first frame of a clip — click it to play the generated video (hosted on the static_videos dataset). The first frame and prompt for every clip also live in assets/: under assets//, frame_N.jpg and prompt_N.txt correspond to grid cell N (left→right, top→bottom).

🚗 Autonomous Driving

First frames + prompts: assets/av/.

🤖 Robotics

First frames + prompts: assets/robotics/.

🌍 General Physics

First frames + prompts: assets/general/.

Results

Kandinsky WM 1.0 on three physical-AI video benchmarks (our row in bold). RBench and Physics-IQ were run with Qwen3-VL prompt enhancers — noted above each table, with the scripts under prompt_enhancers/. Sizes are total parameter counts where publicly disclosed (MoE models note active params; proprietary/undisclosed left as —).

PAI-Bench-G (Physical AI Bench — Generation)

Leaderboard. Column abbreviations follow the PAI-Bench-G dimensions — see the leaderboard for exact definitions. Run without prompt enhancement.