NVIDIA Launches Open-Source 3D CT Analysis Model with Radiologist-Level Reasoning
NV-Reason-CT combines 3D vision transformers with chain-of-thought reasoning for volumetric medical imaging
1Volumetric imaging breakthrough
NV-Reason-CT addresses a critical gap in medical AI by specializing in 3D CT analysis, where traditional 2D models fail to capture spatial relationships. The model processes full 192³ voxel volumes at 2mm resolution using a 3D ViT encoder, preserving anatomical continuity across 300-600 axial slices. This architectural choice enables holistic reasoning about structures, mirroring how radiologists scroll through studies. The system represents NVIDIA's extension of their successful NV-Reason-CXR methodology into volumetric imaging.
2Radiologist-aligned architecture
The model combines a Qwen3.5-4B language model with specialized 3D vision transformers (Primus/Colipri), all weights retrained end-to-end on CT data. Unlike flattened slice approaches, it maintains 3D spatial relationships through 13,824 vision tokens with 3D grid coordinates. The LLM incorporates 3D MRoPE to understand spatial relationships between tokens. This design allows systematic anatomical review, calibrated uncertainty expression, and structured conclusions - mimicking radiologist thought processes rather than acting as a simple classifier.
3Two-stage training pipeline
NV-Reason-CT undergoes supervised fine-tuning on 550,000 structured QA examples from chest/abdominal CTs, including radiologist reasoning annotations. Training data combines CT-RATE, NIH datasets, and CancerVerse with synthetic reasoning examples. A second reinforcement learning phase uses Group Relative Policy Optimization (GRPO) to refine clinical accuracy. The anatomy-aware reward function evaluates correctness per anatomical region rather than globally, improving diagnostic calibration across diverse findings.
4Benchmark outperformance
On the CT-RATE benchmark, NV-Reason-CT achieves 0.614 Macro-F1 and 0.871 Macro-AUROC scores, surpassing all published 3D contrastive models (VoxelFM, Pillar-0), fused 2D/3D systems (ClinFusion-8B), and slice-based models (MedGemma 1.5). NIH radiologists validated the clinical plausibility of its reasoning traces, noting the step-by-step thinking enables trust. This marks the first open model achieving competitive CT classification and report generation simultaneously.
5Clinical workflow integration
The model generates structured reports with reasoning traces that radiologists can follow and verify. NIH's Dr. Baris Turkbey confirmed the system reflects actual CT review processes. Companies like Aidoc and Rad AI can integrate NV-Reason-CT into radiology workflows and PACS systems. The open foundation allows fine-tuning for specific applications while maintaining transparency - critical for clinical adoption where auditability matters more than black-box predictions.
6Technical implementation
Available on Hugging Face, the model processes DICOM/NIfTI inputs through a 3D ViT encoder and reasoning-trained LLM. The GitHub repository provides inference scripts, training configurations, and post-training recipes. A sample implementation loads the model in PyTorch, processes chest CT volumes, and generates chain-of-thought reasoning with anatomical region specificity. The system supports multi-turn conversations about findings, enabling interactive clinical review.
7Medical AI ecosystem
NV-Reason-CT integrates with NVIDIA's medical AI suite including NV-Generate-CTMR for synthetic data and NV-Segment-CTMR for segmentation. Together they form an end-to-end radiology pipeline from data generation to diagnostic reasoning. The open approach allows researchers to study chain-of-thought in medical imaging and companies to develop specialized applications. This ecosystem positioning helps address the data-density challenge in 3D medical imaging AI.
8Specialized 3D challenges
Volumetric CT presents unique AI challenges: through-plane continuity, anatomical context across spatial dimensions, and clinical reasoning depth. Traditional 2D encoders fail to reconstruct 3D relationships from independent slices. NV-Reason-CT's architecture specifically addresses these through dedicated 3D processing and radiologist-aligned reasoning training. The model's ability to explain findings systematically addresses the trust gap in medical AI adoption.
9Future directions
As an open research foundation, NV-Reason-CT enables community development of specialized CT applications. Potential extensions include pediatric adaptations, trauma protocols, and longitudinal change detection. The model's reasoning transparency could help address regulatory concerns about AI explainability in healthcare. By combining technical innovation with clinical usability, NVIDIA positions this work at the forefront of volumetric medical AI development.