How Copilot Code Review Was Actually Improved
Adjusting the instructions, not the tools, led to a 20% cost reduction.
1Root of the Problem
When GitHub Copilot's code review tool switched to better-maintained shared tools, performance dropped unexpectedly. Review costs increased and fewer issues were caught. The issue wasn't the tools—it was the instructions guiding them.
2Legacy System Design
The old system used custom code exploration tools tailored for earlier model behaviors. These tools made fewer calls but bundled extra context, which became inefficient as models evolved.
3New Tools Integration
Integrating shared CLI tools (grep, glob, view) caused the agent to act like a general coding assistant rather than a reviewer. This led to unnecessary broad searches and inflated costs.
4Performance Drop Reason
The agent began scanning entire repos instead of focusing on PR changes. Each tool call added redundant tokens to the context window, making reviews unfocused and costly.
5Human Review Process
Human reviewers start from PR diffs, asking targeted questions with minimal context. The agent needed to emulate this—narrow searches, no irrelevant code—to be efficient.
6Rewriting Instructions
The team redesigned prompts to make the agent behave like a reviewer: focused investigations, minimal context retention. This aligned workflows with actual review needs.
7Results
Post-revision, average review cost dropped by 20% while maintaining quality. Proving that tool usage matters as much as the tools themselves.
8Assistant vs. Reviewer
Coding assistants explore broadly; reviewers investigate narrowly. Copilot's new instructions enforced this distinction, eliminating wasteful browsing loops.
9Shared Infrastructure Benefits
Shared CLI tools serve multiple Copilot products, amplifying improvement impacts. But each product requires tailored instructions for optimal performance.
10Key Takeaway
This case highlights that AI tool efficacy depends heavily on usage instructions. With the right guidance, the same tools can deliver radically better outcomes.
Original article: GitHub Engineering — Copilot Kod İncelemesi Nasıl Geliştirildi?