# Accelerating FLUX 3 Image Generation on NVIDIA Blackwell GPUs Optimizing diffusion model inference using TensorRT LLM VisualGen and Blackwell architecture features. - **Source:** [Nvidia Developer Blog](https://developer.nvidia.com/blog) - **Published:** 2026-11-06 - **Topic:** [technology](https://brifing.si/c/technology) - **Original article:** - **On Brifing:** https://brifing.si/s/07555ad8-96ca-4377-889a-ce9e9cac58ef ## FLUX 3 Model Architecture FLUX 3 Image Klein utilizes a Qwen3-VL-4B-Instruct model for text conditioning, paired with a diffusion transformer and an autoencoder. The system processes text, noisy target images, and reference inputs independently before merging them via joint attention. This architecture supports both Base and distilled checkpoints to balance latency and output fidelity requirements. ## Blackwell GPU Optimization The TensorRT LLM VisualGen runtime leverages NVIDIA Blackwell hardware to accelerate inference. By utilizing NVFP4 quantization, the system achieves up to 1.62x faster text-to-image and 1.58x faster image-to-image generation compared to BF16. These optimizations focus on denoising, which represents the most computationally intensive phase of the image generation pipeline on B200. ## PrimsTS GEMM Kernels VisualGen employs FlashInfer’s PrimsTS kernels, which utilize CUTLASS primitives for task scheduling. These kernels are specifically tuned for FLUX 3 tensor shapes, combining matrix multiplication with fused epilogues. This approach integrates SwiGLU operations and query, key, and value projections with normalization and rotary position embeddings, delivering 1.02–1.10x faster generation speeds. ## Quantized Attention Strategies Attention mechanisms often consume 60% of GPU kernel time during high-resolution generation. VisualGen addresses this via cuDNN FP8 or VC-Attention FP8. Using a repair budget of 0.005, VC-Attention FP8 provides a 1.18x speedup for Base models at 2048x2048 resolution, maintaining high perceptual similarity scores as measured by the LPIPS metric. ## Sol-Attention Sparse Processing Sol-Attention reduces computation by introducing sparsity into the diffusion transformer. By configuring 12 to 16 initial dense steps, users can trade off between image fidelity and speed. At 2048x2048, this method delivers up to 1.30x faster denoising and decoding, providing a flexible mechanism to optimize performance for specific production visual workflows. ## Cache-DiT Latency Reduction Cache-DiT reuses intermediate transformer outputs across denoising steps without requiring model retraining. By setting a residual-difference threshold of 0.24, the system nearly halves the latency for Base models at 1024x1024 and 2048x2048 resolutions. This technique significantly lowers the computational overhead while preserving image quality for complex generation tasks in production. ## Multi-GPU Scaling Performance VisualGen scales FLUX 3 using classifier-free guidance and Ulysses sequence parallelism. On eight B200 GPUs, median text-to-image latency for the Base model dropped from 14.90 to 3.00 seconds. Distilled models showed similar improvements, with denoising speedups reaching 6.01x, demonstrating effective utilization of distributed hardware resources for large-scale image generation workloads. ## CUDA Graphs Integration At lower resolutions, CUDA Graphs minimize kernel-launch overhead, which becomes a bottleneck as work per GPU decreases. In 1024x1024 tests on eight B200s, enabling CUDA Graphs reduced Base latency from 1.823 to 0.809 seconds. This integration is essential for maintaining high throughput when processing smaller image batches or high-frequency generation requests. ## Cumulative Performance Gains Combined optimizations including compilation, NVFP4, Sol-Attention, and Cache-DiT yield significant end-to-end improvements. On a single B200, these steps resulted in a 3.46x speedup for text-to-image and a 4.45x speedup for image-to-image generation. These benchmarks confirm that systematic pipeline optimization is critical for deploying high-parameter diffusion models in real-time professional environments. ## Deployment and Configuration Developers can implement these optimizations using YAML configurations within the TensorRT LLM framework. The process involves setting the variant to nvfp4 and adjusting attention or cache parameters as needed. The provided Docker environment includes the necessary PrimsTS kernels, allowing users to replicate these performance results on Blackwell-based data center servers. ## Hardware Platform Versatility While benchmarks focus on B200 data center GPUs, optimizations also extend to the NVIDIA RTX PRO 6000 Blackwell Server Edition. On this platform, the combination of compilation, fused kernels, and caching delivered speedups between 1.73x and 3.25x. This ensures consistent performance gains across different Blackwell-powered hardware configurations for various visual production needs. ## Future Workflow Considerations Reported latencies exclude prompt upsampling, which should be executed separately before inference. As diffusion models continue to evolve, the integration of specialized runtimes like VisualGen becomes vital for managing inference costs. Future developments will likely focus on further refining sparse attention and quantization techniques to push the boundaries of generative speed. ## Sources covering this story - https://developer.nvidia.com/blog/speed-up-image-generation-with-flux-3-image-klein-on-nvidia-blackwell-gpus/ — --- _Summary written by Brifing from the original reporting. Copyright in the original article belongs to its publisher; read the full piece at the link above._