AesCode

AesCode generates information-rich visual artifacts such as slides, posters, and dashboards as HTML/CSS. The output remains structured, editable, and verifiable, but the task poses a distinct challenge: code models cannot see how layout, hierarchy, and color come together on the canvas.

Image generators offer the opposite strength. They compose visually compelling pages but often misrender text, numbers, and logical relationships. AesCode uses an image generated from the same prompt as an aesthetic reference while following the prompt for the required content.

Reference input alone does not solve the problem. Off-the-shelf vision-language models may copy hallucinated content or ignore the reference layout. AesCode separates semantic requirements from visual cues through graph-structured supervision and decoupled cross-modal rewards.

AesCode-32B starts from Qwen3-VL-32B-Instruct and is trained with cold-start SFT followed by GDPO across seven reward channels. It is the larger released checkpoint and leads the aggregate benchmark results.

• 📄 Paper: AesCode: Aesthetic Code Generation with Decoupled Cross-Modal Rewards
• 💻 Code: https://github.com/microsoft/AesCode
• 🤗 Companion model: microsoft/AesCode-8B

Intended Use

AesCode is designed to generate information-rich visual artifacts as structured HTML/CSS. It is suited for creating editable slides, posters, dashboards, and reports, as well as for research on multimodal code generation and verifiable visual design.

Results

We evaluate on 300 infographic samples. Rule averages the programmatic Text, Bound, and Chart checks. Visual averages the Content, Layout, and Style checklist dimensions. Overall is the mean of Rule and Visual. Scores are percentages averaged over three generations per prompt with no selection.

Ref. indicates whether the model receives an image-generated reference together with the prompt. Bold marks the best score in each column and underline the second best.

AesCode-32B improves Visual over its reference-conditioned Qwen3-VL-32B backbone by 22.4 points and Overall by 24.8. It leads the comparison on Overall at 85.89 and reaches Rule 94.33, compared with 85.90 for GPT-5.5 and 88.04 for Claude Opus 4.8.

Two results stand out. Table/Chart at 90.37 leads all models: requiring tables to use HTML table structures and charts to use ECharts constrains the solution space to executable, directly verifiable implementations. Boundary at 97.27 reflects a sharp reduction in canvas overflow: a severe boundary failure recurs on 4.3% of the 300 samples, against 34.7% for GPT-5.5.

Style is the shared ceiling for every system. It awards credit only when a design needs no further visual revision before delivery, and no model clears 60.

Usage

The model follows the Qwen3-VL chat interface and requires transformers>=4.57. Supply a detailed content prompt and, when available, a reference image for layout and style guidance. The weights are bf16; plan for roughly 65 GB of accelerator memory for the parameters alone.

from transformers import AutoProcessor, AutoModelForImageTextToText

model_id = "microsoft/AesCode-32B"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id, dtype="auto", device_map="auto"
)

messages = [{
    "role": "user",
    "content": [
        {"type": "image", "image": "reference.png"},
        {"type": "text", "text": ""},
    ],
}]

inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True,
    return_dict=True, return_tensors="pt",
).to(model.device)

out = model.generate(
    **inputs, do_sample=True, temperature=0.8, top_p=0.95, max_new_tokens=12000
)
print(processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

For batched generation, serve with vLLM and allow two images per prompt:

vllm serve microsoft/AesCode-32B --tensor-parallel-size 4 \
  --limit-mm-per-prompt image=2 --max-model-len 24576

Reported scores use three rollouts per prompt at temperature 0.8, top-p 0.95, up to 12,000 output tokens and a 24,576-token context.

The model emits a complete HTML document. Tables use HTML table structures and charts use ECharts specifications, keeping both directly inspectable and verifiable.

The reference image is optional. Reference-conditioned training internalizes visual planning into the policy. Measured on AesCode-8B, withholding the reference at inference costs only 1.00 Visual point, against 19.55 for the Qwen3-VL-8B-Instruct backbone and 10.04 for GPT-5.5. Quality is still highest when a reference is supplied.

The repository provides the training prompt template and a pipeline that turns a short idea into the