ALoDLM

Paper: ALoDLM: Adaptively Looped Diffusion Language Models · Project page: https://alo-dlm.github.io/

ALoDLM is a family of diffusion language models developed at Amazon, available as ALoDLM-1.7B and ALoDLM-8B and initialized from the corresponding Qwen3 backbones. The models generate text and code by repeatedly applying shared transformer layers, preserving unresolved tokens' latent states, and adaptively allocating computation across token positions.

Training data includes mathematical problems and worked solutions, programming tasks and code solutions, and instruction-formatted text.

Models: ALoDLM-1.7B · ALoDLM-8B Code: amazon-science/ALoDLM License: Creative Commons Attribution-NonCommercial 4.0 International

Research overview

ALoDLM: Adaptively Looped Diffusion Language Models studies how to allocate computation while generating multiple tokens in parallel. In masked diffusion, some unknown positions can be resolved from the available context, while others benefit from further refinement. A fixed-depth denoiser applies the same computational depth to every position. Our research explores whether preserving and refining latent states within each denoising step can improve this allocation.

ALoDLM adds adaptive recurrence to shared transformer layers. Tokens that commit early provide resolved context for the remaining positions. Unresolved tokens retain their hidden states and refine them through additional recurrent passes. Predictive confidence controls token commitment, while a learned halting policy determines when to end the recurrent loop.

The work develops three components:

• Persistent latent refinement: reuse shared layers to refine unresolved representations across recurrent passes.
• Learned computation allocation: treat token-wise computation schedules as latent variables and derive a conditional negative evidence lower bound for jointly learning token prediction and computation allocation.
• Inference-time control: use the gate threshold q and entropy threshold τ to explore the trade-off between recurrent refinement and parallel token commitment.

Decoding overview from the paper. Committed tokens provide discrete context while unresolved tokens retain latent states for further refinement. The expanded schematic shows the recurrent core and its readouts; prefix and suffix transformer blocks are optional components of the architecture.

Research performance

Benchmark profile from the paper's 8B comparison. Each spoke represents one benchmark; farther outward indicates a higher score. Compare models along each spoke, since the axes use different score ranges. The table in Evaluation reports both model sizes.

Inference efficiency from the paper's 8B experiments. The plot compares GSM8K accuracy with single-stream generation throughput on one NVIDIA B200 GPU, using the optimized inference engine named for each model. The throughput measurements use parallel commitment, a separate operating mode from the quality comparison below. They do not report 1.7B throughput or establish the speed of the portable decoder.

Intended use

The models support noncommercial research on diffusion language modeling, adaptive computation, mathematical reasoning, and code generation, including the effects of changing inference-time computation.

Downstream use requires evaluation for the intended application. The reported benchmarks do not establish readiness for autonomous operation or decisions with significant consequences.

Evaluation

The following table reproduces all model columns and scores from Table 1 in ALoDLM: Adaptively Looped Diffusion Language Models. All scores are percentages.

The paper follows the OpenCompass evaluation protocol adopted by SDAR, using each model's native chat template, greedy token selection, and a 4,096-token generation limit. Both ALoDLM models use a 16-token window, with mean cumulative halting threshold q = 0.4 for 1.7B and q = 0.5 for 8B. Each recurrent pass commits one token, with entropy-based selection disabled (τ is inactive); successive recurrent passes can commit multiple tokens within a denoising step.

Qwen3 models are autoregressive (AR) baselines; SDAR, LLaDA, Dream, Fast-dLLM-v2, and WeDLM are diffusion language model (DLM) baselines. As in the paper, bold marks the best overall score within each size group, and underlining marks the best non-AR score. ALoDLM columns are shaded in blue in the rendered table.