Diffusion LLMs' Flexibility Trap: Arbitrary-Order Generation Hurts Reasoning — Fix Reaches 89.1% on GSM8K

Researchers at Tsinghua University and Alibaba Group have measured that arbitrary-order generation — the signature architectural feature of diffusion language models — actively degrades reasoning performance: on HumanEval with k=1,024 samples, 21.3% of problems are solvable exclusively under autoregressive ordering, while the reverse holds for a mere 0.6%. The finding earned the Outstanding Paper prize at ICML 2026 — one of two selected from 53 candidates — and the remedy is already public: replacing arbitrary-order reinforcement learning with standard GRPO on fixed left-to-right sequences (JustGRPO) reaches 89.1% on GSM8K while fully preserving parallel-decoding throughput.

How Arbitrary Order Undermines Reasoning

Diffusion language models (dLLMs) have attracted enterprise interest for a specific capability: parallel token generation, which delivers throughput gains over sequential autoregressive (AR) decoding. The foundational claim of the dLLM research programme has been that freedom to generate tokens in arbitrary order — the mechanism enabling parallelism — also benefits reasoning by allowing richer exploration of solution space. Zanlin Ni, Shenzhi Wang, and nine co-authors from Tsinghua and Alibaba tested that claim directly.

Their analysis centres on what they term "forking tokens": high-entropy connectives — Therefore, Thus, Since, Let, Solving, Notice — that steer logical trajectories into distinct reasoning branches. Under arbitrary-order generation, models systematically defer these tokens. By the time the model returns to fill them, surrounding context is already written and entropy at those positions has collapsed. The paper identifies this as a structural mechanism: the very act of deferring high-uncertainty tokens eliminates the branching potential that makes chain-of-thought reasoning productive.

The coverage data is unambiguous. On HumanEval with k=1,024 samples, 21.3% of problems are solvable exclusively under autoregressive ordering; the converse — problems reachable exclusively through arbitrary order — accounts for 0.6%. Arbitrary-order outputs are, in the researchers' framing, nearly a strict subset of autoregressive ones. The flexibility dLLMs advertise as a reasoning asset functions as a constraint on solution diversity.

The team confirmed the mechanism is continuous. Sweeping semi-autoregressive block size B across a range of ordering freedom, they found a monotonic degradation: Pass@k falls consistently as B grows, across k ∈ {8, 32, 128} and all four benchmarks (GSM8K, MATH-500, HumanEval, MBPP). The effect holds at every tested parallelism level and generalises across task domains — code, mathematics, and grade-school arithmetic alike.

JustGRPO addresses the training-time cause directly. The method applies standard Group Relative Policy Optimization to the autoregressive factorisation during RL training, leaving the model architecture and diffusion-based inference procedure untouched. Inference remains fully parallel. Trained for 125 steps on 16×H100 GPUs with learning rate 5×10⁻⁶ and group size 16, JustGRPO reaches 89.1% on GSM8K and 45.1% on MATH-500. Prior diffusion-specific RL methods score substantially lower: ESPO at 82.3% / 39.0%, SPG at 86.1% / 40.0%, d1 at 81.1% / 38.6%, and GDPO at 82.8% / 39.6%.

The parallel-decoding throughput advantage compounds rather than erodes. Measured with the Entropy Bounded Sampler at increasing parallelism levels, JustGRPO's accuracy margin expands as throughput rises: on MBPP, the gap grows from +10.6% at one token per step to +25.5% at approximately five tokens per step. A practical variant, JustGRPO-Fast, concentrates probability-ratio computation at the top-25% highest-entropy positions, eliminating 75% of additional forward evaluations while matching or exceeding the standard variant on wall-clock accuracy comparisons.

Why Enterprises Evaluating dLLMs Need These Numbers

Diffusion language models are entering enterprise procurement conversations on a single architectural pitch: faster inference through parallel generation. Mercury-class commercial models and Gemini Diffusion both position throughput as their primary differentiation. The speed claim is accurate — JustGRPO validates and extends it. What the ICML 2026 Outstanding Paper committee — eleven members, 53 initial candidates, 22 shortlisted — resolved in selecting this paper among two Outstanding Papers is that the reasoning cost of arbitrary-order generation is an empirically verified structural finding, treated by the field's most competitive annual venue as a settled question.

The practical consequence for enterprise evaluation is direct. An 8-percentage-point gap between JustGRPO (89.1%) and d1 (81.1%) on GSM8K is a gap that arbitrary-order RL training actively creates. For reasoning workloads — multi-step financial analysis, code generation at scale, legal document review — that gap carries measurable production cost. The forking-token analysis gives evaluators a diagnostic framework: measure entropy at logical connectives during generation and compare arbitrary-order versus AR-order fill rates. A model deferring high-entropy forking tokens carries a measurable solution-diversity deficit regardless of its architecture label.

The study uses LLaDA-Instruct as its base model. Monotonic degradation across four benchmarks and the coverage-overlap analysis (21.3% versus 0.6%) provide grounds for treating the finding as structural to arbitrary-order decoding broadly. Replication across MDLM, Plaid, and Mercury-class architectures remains an open empirical question — and one that downstream procurement decisions should require vendors to answer before accepting reasoning benchmarks at face value.

The R&D Roadmap Question

The specific question for any team with a dLLM on its roadmap: does your reasoning evaluation assume arbitrary-order generation, and does your RL fine-tuning preserve that assumption? Replacing arbitrary-order training with JustGRPO's 125-step fix — code and models are public on arXiv — recovers the full parallel-decoding throughput advantage and adds an 8-point reasoning lift. The architectural bet that dLLMs sell on turns out to be separable from the inference-speed bet that makes them worth buying. Procuring the speed advantage requires retaining the training constraint that restores reasoning.