# An Engineer's Guide to Enhancing AI Skills Testing and optimizing AI agent performance for reliable coding. - **Source:** [Pinterest Engineering](https://medium.com/pinterest-engineering) - **Published:** 2026-05-12 - **Topic:** [Data & AI](https://brifing.si/c/data-ai) - **Original article:** - **On Brifing:** https://brifing.si/s/8ce1d682-2729-4c91-bc2d-45be244dacfc ## The Reliability Challenge of AI Agents Pinterest engineers encountered inconsistent results when using AI agents to load a custom iOS architecture skill. The failure to reliably load skills during code writing caused issues in architectural reviews. This highlighted the need for a systematic testing process to measure AI agent reliability. ## Building a Test Harness The core of the testing process is a Bash script that manages automated tests. It runs both positive scenarios (testing correct skill loading) and negative scenarios (catching false positives). Given the nondeterministic nature of AI agents, tests were repeated multiple times to ensure reliable results. ## Metrics and Analysis Test results were evaluated using three key metrics: core success rate, false positive rate, and overall accuracy. Initial results showed 73% accuracy for Codex and 62% for Claude - insufficient for critical engineering workflows. These findings clearly demonstrated the need for optimization. ## Optimization Techniques Various techniques were tested to improve skill loading rates. Adding more contextual information to frontmatter descriptions, using aggressive language commands, and including skill tables in AGENTS.md all showed improvements. These methods proved particularly effective for Codex users. ## The Combination Effect Applying multiple optimization techniques together compounded improvements for Codex, but not for Claude. Surprisingly, asking the agents to further optimize the improvements actually decreased performance. This showed that optimization requires careful implementation. ## Conclusions and Recommendations Systematic testing and optimization are crucial for improving AI agent reliability. However, engineers must also use clear, detailed prompts. Pinterest's experience demonstrates that combining both approaches is essential for maximizing the effectiveness of AI-assisted coding tools. ## Sources covering this story - https://medium.com/pinterest-engineering/an-engineers-guide-to-better-ai-skills-implementing-a-testing-process-to-optimize-agent-a000c9c9abcd?source=rss----4c5a5f6279b6---4 — --- _Summary written by Brifing from the original reporting. Copyright in the original article belongs to its publisher; read the full piece at the link above._