ORL Enhances Diffusion Language Model Inference Speed
New on-policy residual learning method achieves massive throughput gains on diffusion-based language models.
1Diffusion Language Model Breakthrough
The research team introduced On-Policy Residual Learning (ORL) to accelerate diffusion language models (dLLMs). By enabling models to refine tokens in parallel rather than sequentially, the approach overcomes traditional autoregressive bottlenecks. This method significantly boosts generation speeds while maintaining or exceeding the quality of standard models across various benchmarks and architectures.
2On-Policy Residual Learning Mechanism
ORL improves convergence by training directly on fresh trajectories generated by the current model. By replaying exact residual states encountered during inference, it eliminates the distribution mismatch common in offline training. This ensures the model effectively carries forward previous predictions as context, allowing for faster and more accurate token refinement steps.
3Adjacent Entropy Contraction Technique
To optimize convergence speed, researchers implemented Adjacent Entropy Contraction. This technique encourages each denoising step to reduce uncertainty relative to the previous iteration. By gating the loss function, the model avoids wasting compute on already-confident predictions, allowing it to finalize more tokens per forward pass without compromising the overall generation quality.
4Custom Inference Engine dlmrt
The team developed dlmrt, an inference engine specifically optimized for low-latency, single-request diffusion decoding. By capturing the entire denoising loop in a CUDA graph, it minimizes CPU-GPU synchronization overhead. The engine supports NVFP4 inference on Blackwell GPUs, delivering 1.2 to 1.6 times faster performance than standard vLLM or SGLang frameworks.
5Benchmark Performance Metrics
On a single RTX 5090, the LLaDA-2.1-mini model with ORL achieved 2,788 tokens per second. Meanwhile, the DiffusionGemma 26B MoE reached 7,708 tokens per second on 8x B200 GPUs. These results highlight the system's capability to handle high-throughput demands while maintaining competitive accuracy levels compared to traditional speculative decoding methods.
6QwenDLM-4B Model Improvements
The QwenDLM-4B model demonstrated significant gains, with end-to-end throughput increasing by approximately 6.9 times. HumanEval pass@1 scores improved from 82.9% to 89.6%, proving that speed optimizations do not necessitate a trade-off in reasoning capabilities. The model effectively utilizes residual-state capture and compute-balanced packing to accelerate training steps by up to 5.2 times.
7Comparison With Speculative Decoding
In direct comparisons on a single H100, QwenDLM-4B with ORL reached 1,298 tokens per second. This outperforms DFlash at 746 tokens per second and EAGLE3 at 405 tokens per second under relaxed verification. These figures confirm that ORL provides a superior alternative for single-request decoding scenarios where latency is the primary performance constraint.
8Open Source Research Impact
The project is fully open-sourced, including models, training code, evaluation pipelines, and the dlmrt engine. By providing these tools, the researchers invite community feedback on local inference bottlenecks. This contribution aims to advance the adoption of diffusion-based architectures for real-time applications, shifting focus toward efficient, low-latency deployment on consumer-grade hardware.
Original article: r/MachineLearning — ORL Enhances Diffusion Language Model Inference Speed