Model reference · open weights
HY-OmniWeaving is an open-weight video model from tencent. HY-OmniWeaving (BF16) weighs 55.3 GB; the smallest configuration that runs it is L40S 48 GB.
For self-hosting, compare this build's requirements below with the GPU you plan to rent.
Summary of the tencent/HY-OmniWeaving model card, 2026-10-01
What it is
| Released by | tencent |
|---|---|
| Released | 2026-03-31 |
| VRAM | 55.3 GB for the weights |
What it runs on
| Card | Weights | Memory |
|---|---|---|
| RTX 3060 12 GB … RTX 5090 32 GB | does not fit | |
| L40S 48 GB | fits (encoders offloaded) | 44.0 GB |
| A100 80 GB | fits | 78.2 GB |
| H100 80 GB | fits | 78.1 GB |
| RTX PRO 6000 Blackwell 96 GB | fits | 93.8 GB |
| DGX Spark (GB10) 128 GB unified | fits | 107 GB |
| H200 141 GB | fits | 138 GB |
| B200 180 GB | fits | 176 GB |
From the model card
Kaihang Pan*1,2, Qi Tian*2, Jianwei Zhang2, Weijie Kong2, Jiangfeng Xiong2, Yanxin Long2, Shixue Zhang2, Haiyi Qiu1, Tan Wang3, Zheqi Lv1, Yue Wu§2, Liefeng Bo2, Siliang Tang§1, Zhao Zhong†2
Work done during Kaihang Pan's internship at Tencent Hunyuan
We propose OmniWeaving, an omni-level video generation model featuring powerful multimodal composition and reasoning-informed capabilities. By leveraging a massive-scale pretraining dataset that encompasses diverse compositional and reasoning-augmented scenarios, OmniWeaving learns to temporally bind interleaved text, multi-image, and video inputs while acting as an intelligent agent to infer complex user intentions for sophisticated video creation. Furthermore, we introduce IntelligentVBench, the first comprehensive benchmark designed to rigorously assess next-level intelligent unified video generation. Extensive experiments demonstrate that OmniWeaving achieves SoTA performance among open-source unified models.
Following the paper, OmniWeaving is built as an integrated MLLM + MMDiT + VAE framework for unified free-form video generation. The MLLM serves as the semantic parser for interleaved text, images, and video inputs, mapping them into a high-level semantic space and forwarding its hidden states through an MLP connector. The VAE acts as the visual tokenizer, compressing visual inputs into low-level latents, while the MMDiT uses these semantic conditions together with latent noise to generate semantically aligned, high-fidelity videos.
On this basis, we further introduce two extra improvements tailored for advanced reasoning and composition.
Figure 1. Overview of the OmniWeaving architecture, which consists of an MLLM for multimodal understanding and an MMDiT for generation.
OmniWeaving is flexible in its input and output configurations, supporting a wide range of unified video generation tasks:
git clone https://github.com/Tencent-Hunyuan/OmniWeaving
cd OmniWeaving
OmniWeaving is built upon HunyuanVideo-1.5. The way to install dependencies is similar to HunyuanVideo-1.5. Specifically, you should install basic dependencies:
pip install -r requirements.txt
Additionally, install the attention libraries as needed (we use Flash Attention in practice):
Flash Attention: Install for faster inference and reduced GPU memory consumption. See Flash Attention for details.
Flex-Block-Attention: Required only for sparse attention to achieve faster inference:
git clone https://github.com/Tencent-HuQuoted from the model card on Hugging Face. The full card is behind the Hugging Face link above.
Running it yourself
Rent a machine by the hour. ComfyUI is installed on it. Open ComfyUI through the tunnel and load the workflow from the model's card on Hugging Face; choose this model's file in its loader.
# on your rented machine (the ssh line is on its page in the console)
# get REPO FILE FOLDER: one file into /workspace/models/FOLDER, where ComfyUI loads it from
get() { hf download "$1" "$2" --local-dir /workspace/hf-files && mkdir -p "/workspace/models/$3" && mv "/workspace/hf-files/$2" "/workspace/models/$3/$4"; }
# the model (51.2 GB)
get tencent/HY-OmniWeaving transformer/diffusion_pytorch_model.safetensors diffusion_models
get tencent/HY-OmniWeaving text_encoder/ckpt/text_encoder_model.safetensors text_encoders
get tencent/HY-OmniWeaving vae/diffusion_pytorch_model.safetensors vae
start-comfyui# on your computer, in a second terminal: ComfyUI in your browser at http://localhost:8188
# HOST, PORT and BOOKING are your machine's, from its page in the console
ssh -o HostKeyAlias=axforge-BOOKING -o StrictHostKeyChecking=accept-new -L 8188:localhost:8188 dev@HOST -p PORT