# Developing Nemotron 3.5 Lightning NVFP4

High-performance model compression with NVIDIA Model Optimizer

- **Source:** [Nvidia Developer Blog](https://developer.nvidia.com/blog)
- **Published:** 2026-08-17
- **Topic:** [Data & AI](https://brifing.si/c/data-ai)
- **Original article:** <https://developer.nvidia.com/blog/developing-nemotron-3-5-lightning-nvfp4-with-qad-using-nvidia-model-optimizer/>
- **On Brifing:** https://brifing.si/s/d27843c9-b05c-459e-a0f1-5ffeb4a31062

## What is Nemotron 3.5 Lightning NVFP4?

Nemotron 3.5 Lightning NVFP4 is an AI model developed by NVIDIA. Using aggressive compression techniques, the model reduces its size from 66 GB to 22 GB while maintaining the accuracy of a BF16 baseline. This compression increases processing speed by 4x and significantly reduces memory usage.

## The Quantization-Aware Distillation (QAD) Process

QAD is a two-stage process. In the first stage, a low-precision student model is created from a full-precision teacher model. In the second stage, the student model is trained against the frozen teacher model using KL divergence loss, recovering almost all accuracy lost from aggressive quantization.

## The Role of NVIDIA Model Optimizer

NVIDIA Model Optimizer is a tool that facilitates the QAD process. It provides a workflow that includes dynamic and frozen scale selection, training configuration, and checkpoint export. This enables the widespread adoption of memory- and compute-efficient LLM deployment.

## Comparison of PTQ and QAD

PTQ is a common method that covers most needs. However, for higher throughput and tighter memory usage, QAD is preferred. QAD enables aggressive quantization settings that PTQ alone cannot achieve while maintaining accuracy.

## Performance of Nemotron 3.5 Lightning NVFP4

Experiments on multiple checkpoints of Nemotron 3.5 Lightning NVFP4 showed that QAD provides higher median accuracy recovery compared to PTQ. This proves QAD's ability to maintain high quality even with aggressive quantization settings.

## Benefits of Model Compression

Model compression allows for more efficient use of memory and computational resources. This is particularly advantageous in the deployment of large language models. Nemotron 3.5 Lightning NVFP4 maximizes these benefits, offering a high-performance and efficient model.

## Details of the QAD Process

In the QAD process, every forward pass of the student model runs through simulated quantization. This allows the model to account for the quantization noise it will encounter during inference. Simultaneously, the student model is trained to match the teacher model, ensuring it learns the full behavior of the model.

## Training Nemotron 3.5 Lightning NVFP4

Nemotron 3.5 Lightning NVFP4 is trained on the NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 model. This model serves as the teacher, and the student model is created by running PTQ on the same model. This ensures the student model has a well-matched teacher to learn from.

## PTQ Recipes and Trials

During the development of Nemotron 3.5 Lightning NVFP4, various PTQ recipes were tested. These recipes enabled more aggressive compression and higher processing speeds. These trials also proved QAD's ability to recover accuracy loss.

## The Future of QAD

QAD will play a significant role in the compression of AI models. This technique enables more efficient use of memory and computational resources, facilitating the broader adoption of large language models. Nemotron 3.5 Lightning NVFP4 represents a significant step in this field.

## NVIDIA's Contribution

NVIDIA plays a crucial role in the compression of AI models by developing tools and techniques that facilitate the QAD process. NVIDIA Model Optimizer stands out as a tool that enables the widespread adoption of this process.

## The Significance of Nemotron 3.5 Lightning NVFP4

Nemotron 3.5 Lightning NVFP4 represents a significant achievement in the compression of AI models. This model offers high performance and efficiency, facilitating the broader adoption of large language models. NVIDIA's contributions in this field promise hope for future developments.

## Sources covering this story

- https://developer.nvidia.com/blog/developing-nemotron-3-5-lightning-nvfp4-with-qad-using-nvidia-model-optimizer/ — 

---

_Summary written by Brifing from the original reporting. Copyright in the original article belongs to its publisher; read the full piece at the link above._