Optimizing Agent Reliability in Chip Design with NVIDIA NeMo Gym
Siemens EDA improves AI agent performance by 40% using systematic artifact verification.
1The Reliability Challenge
AI agents in complex chip design workflows often generate plausible but incorrect artifacts. In multistage engineering, an early setup error propagates silently, leading to faulty downstream outputs. Siemens EDA addressed this by shifting focus from isolated intermediate steps to verifying final Liberty file outputs, ensuring reliability before production adoption in semiconductor design.
2NeMo Gym Implementation
NVIDIA NeMo Gym provides an open evaluation environment to diagnose failure modes and track error propagation. By defining specific tasks, tools, and states, the framework allows engineers to trace complete rollout trajectories. This environment was critical for the Siemens Fuse EDA AI Agent to identify root causes behind subtle characterization errors.
3Standard-Cell Characterization
Standard-cell characterization generates timing, power, and noise models required whenever a foundry issues a new process design kit. The process involves complex SPICE simulations and PVT condition analysis. Because these stages are interdependent, the Solido Characterization Suite agent must maintain high precision to avoid hidden failures in the final library.
4Verification Methodology
The team implemented a domain-specific verifier that inspects final Liberty artifacts against reference data. It checks coverage, validity, and integrity, assigning a zero reward if any gate fails. This rigorous protocol ensures that the agent is evaluated against final results rather than individual tool calls, preventing the acceptance of incorrect thresholds.
5Performance Gains
Using NeMo Gym, the team increased the composite verification score from approximately 55% to 95% across representative benchmarks. This improvement resulted from refining agent skills and Model Context Protocol tool descriptions. The systematic tuning process allowed the agent to handle complex characterization tasks with significantly higher accuracy than the initial baseline.
6Efficiency Improvements
Beyond accuracy, the optimization significantly reduced computational overhead. Agent turns decreased from 155 to 53, representing a 66% reduction. Simultaneously, output tokens dropped from 81K to 29K, a 64% decrease. These metrics demonstrate that better tool descriptions and explicit JSON schemas directly correlate with more efficient and reliable agentic workflows.
7Scaling the Approach
The evaluation method is applicable to broader engineering domains including CAE and EDA. By defining valid final artifacts and building domain-specific verifiers, developers can systematically inspect failures. The process involves repeated rollouts where failure analysis feeds directly into skill revisions, creating a continuous improvement loop for specialized engineering AI agents.
8Strategic Implications
This collaboration highlights the necessity of controlled evaluation loops in industrial AI. By integrating NeMo Gym with the Solido Characterization Suite, Siemens demonstrated that agent reliability is achievable through rigorous verification. This framework serves as a practical blueprint for teams looking to deploy trustworthy, long-running agents in complex, high-stakes engineering environments.