Model reference · open weights
GenFirst is an open-weight image model from ByteDance. GenFirst (BF16) weighs 672 MB; the smallest configuration that runs it is RTX 3060 12 GB.
For self-hosting, compare this build's requirements below with the GPU you plan to rent.
Summary of the ByteDance-Seed/GenFirst model card, 2026-11-04
What it is
| Released by | ByteDance |
|---|---|
| Released | 2026-10-29 |
| VRAM | 672 MB for the weights |
What it runs on
| Card | One image | Memory |
|---|---|---|
| RTX 3060 12 GB | fits | 11.6 GB |
| RTX 4060 Ti 16 GB | fits | 15.4 GB |
| RTX 3090 24 GB | fits | 23.4 GB |
| RTX 4090 24 GB | fits | 23.4 GB |
| RTX 5090 32 GB | fits | 31.0 GB |
| L40S 48 GB | fits | 44.0 GB |
| A100 80 GB | fits | 78.2 GB |
| H100 80 GB | fits | 78.1 GB |
| RTX PRO 6000 Blackwell 96 GB | fits | 93.8 GB |
| DGX Spark (GB10) 128 GB unified | fits | 107 GB |
| H200 141 GB | fits | 138 GB |
| B200 180 GB | fits | 176 GB |
From the model card
Generation Before Reconstruction for Stable End-to-End Latent Generative Modeling.
These weights are released under CC BY-NC 4.0: non-commercial research use only, which is stricter than the MIT licence on the code. See Licence and third-party terms before use.
GenFirst trains a latent VAE jointly with a generative prior without the latent collapse that usually makes such end-to-end training unstable. The method is a two-phase schedule over the prior and entropy weights: phase 1 lets the prior shape the latent geometry, phase 2 drops both weights tenfold so the decoder recovers detail. The result is a latent space that a diffusion prior fits well while remaining a good autoencoder.
This release covers the EiT (diffusion-prior) instantiation. The autoregressive EAR instantiation from the paper is not released.
Two kinds of artifact are published, and they are used together:
| Developed by | Guangting Zheng, Yiyuan Zhang, Tao Yang, Yunpeng Chen, Rui Zhu, Jiajun Deng, Yanyong Zhang |
| Model type | Latent diffusion (flow matching / stochastic interpolant) with a jointly trained VAE |
| Prior architecture | SiT-XL and MMDiT-XL: 28 blocks, width 1152, 16 attention heads (~675M parameters) |
| Text encoder (text-to-image) | CLIP-L, frozen, not modified by us |
| Precision | Weights released in fp32; sampling tested in bf16 |
| Framework | PyTorch |
| Inference code | Required — see Usage. These are raw .pt checkpoints, not diffusers pipelines. |
SiT-XL on a frozen end-to-end VAE. Conditioning is a single integer class
label in [0, 999] following the standard 1000-class label ordering; there is
no text input. Epoch counts are prior-only training epochs.
| Checkpoint | Resolution | VAE | gFID | IS | gFID (guided) | IS (guided) | Download |
|---|---|---|---|---|---|---|---|
| EiT, 80 epochs | 256x256 | f16d32 | 2.79 | 175.4 | 1.38 | 271.3 | checkpoint · weak-20ep |
| EiT, 480 epochs | 256x256 | f16d32 | 1.60 | 216.8 | 0.988 | 270.7 | checkpoint · weak-20ep |
| EiT, 800 epochs | 256x256 | f16d32 | 1.45 | 226.7 | 0.969 | 283.4 | checkpoint · weak-20ep |
| EiT, 80 epochs | 512x512 | f16d32 | – | – | 1.63 | 278.9 | checkpoint |
| EiT, 400 epochs | 512x512 | f16d32 | – | – | 1.23 | 286.1 | checkpoint |
The 512 models reuse the 256 f16d32 VAE unchanged and frozen; only the prior
is trained at 512.
MMDiT-XL/2 with a frozen CLIP-L text encoder, on a frozen end-to-end FLUX
VAE, at 512x512. Scores are overall GenEval and DPG-Bench, with no prompt
rewriting. The two variants differ only in which VAE they were trained on.
| Checkpoint | Resolution | GenEval | DPG-Bench | Download |
|---|---|---|---|---|
| EiT T2I, variant A | 512x512 | 0.8987 | 82.60 | checkpoint · VAE |
| EiT T2I, variant B | 512x512 | 0.8920 | 82.06 | checkpoint · VAE |
MMDiT-XL/2 on a frozen Qwen3-VL ViT autoencoder (--vae-arch qwenvit) shaped end-to-end with a generative prior plus representation supervision (SigLIP or VLM-NLL, Table 9 in the paper).
| Checkpoint | VAE bundle | Supervision | Resolution | rFID ↓ | PSNR ↑ | GenEval ↑ | DPG-Bench ↑ | Linear Probe Top-1 (%) ↑ |
|---|---|---|---|---|---|---|---|---|
shared-latents/siglip/eit-qwenvit-siglip-t2i-512.pt | vae/qwenvit-siglip-b256-s2/ | E2E + SigLIP | 512x512 | 0.39 | 27.64 | 0.90 | 83.76 | 79.25 |
shared-latents/vlm-nll/eit-qwenvit-vlm-nll-t2i-512.pt | vae/qwenvit-nlltrunk-s2/ | E2E + VLM-NLL | 512x512 | 0.46 | 27.19 | 0.89 | 83.49 | 81.69 |
Quoted from the model card on Hugging Face. The full card is behind the Hugging Face link above.
How it works
Running it yourself
Rent a machine by the hour. ComfyUI is installed on it. Open ComfyUI through the tunnel, load the workflow from the model's card on Hugging Face, and choose this file in its Load VAE node.
# on your rented machine (the ssh line is on its page in the console)
# get REPO FILE FOLDER: one file into /workspace/models/FOLDER, where ComfyUI loads it from
get() { hf download "$1" "$2" --local-dir /workspace/hf-files && mkdir -p "/workspace/models/$3" && mv "/workspace/hf-files/$2" "/workspace/models/$3/$4"; }
# the model (641 MB)
get ByteDance-Seed/GenFirst vae/e2e-vae-flux2-sitxl2-auxkl2p5e8-s3-20k/diffusion_pytorch_model.safetensors vae
get ByteDance-Seed/GenFirst vae/e2e-vae-flux2-sitxl2-auxkl2p5e8-s3-20k/e2e-vae-flux2-sitxl2-auxkl2p5e8-s3-20k.safetensors vae
start-comfyui# on your computer, in a second terminal: ComfyUI in your browser at http://localhost:8188
# HOST, PORT and BOOKING are your machine's, from its page in the console
ssh -o HostKeyAlias=axforge-BOOKING -o StrictHostKeyChecking=accept-new -L 8188:localhost:8188 dev@HOST -p PORT