The Economic Benefits of AI-Assisted Code Refactoring
Claude-powered refactoring experiment reveals significant token savings.
1AI-Assisted Code Development
Giles, CTO at Thoughtworks, explores AI-assisted code development processes. He developed a 150,000-line application using Rust, TypeScript, and Terraform. The application features a dynamic web UI, machine learning, and automated deployment. The entire code was written using AI tools like Claude Code and Cursor.
2Growth of the Data Access Layer
During development, the data access layer expanded from 6,000 to 17,155 lines. This growth was due to repetitive HTTP requests and JSON encoding/decoding. The lack of code reuse and class extraction made it a prime target for refactoring.
3Purpose of Refactoring
The primary goal of refactoring is to reduce token consumption for future tasks. Giles measured token costs after each refactoring step by making the same change. This experiment was possible because AI tools do not learn from previous steps.
4Token Savings Results
Refactoring reduced token consumption for the same task from 159,564 to 27,360, an 83% savings. This saving applies to all future changes. The reduction was achieved by the AI's ability to read smaller sections of code.
5Refactoring Steps
The refactoring process involved increasing code reuse and breaking it into smaller files. These steps were not designed to maximize token savings but emerged as a result of typical refactoring practices.
6Claude's Refactoring Capability
Claude struggled to determine appropriate refactoring steps without human intervention. Giles had to manually specify the steps, highlighting the limitations of AI tools in this area.
7Token Cost and Output
Refactoring did not significantly affect output token production. Although output tokens are five times more expensive than input tokens, they are fewer in number. This raises questions about refactoring's potential to reduce output token production.
8Limitations of the Experiment
The experiment was conducted on only a portion of the codebase. It is unclear how much savings would increase if similar refactoring steps were applied to the entire codebase. Additionally, refactoring costs must be considered.
9AI and Code Reading
The success of refactoring depends on AI tools' ability to identify the smallest subset of code necessary. In the experiment, Claude read smaller sections of code with each step, which was key to token savings.
10Future Research
Giles plans to investigate how refactoring affects output token production with more complex sample changes. This could lead to further optimization in AI-assisted code development processes.
11Industrial Applications
This experiment demonstrates how AI-assisted code development can be applied in industrial settings. Token savings could provide significant cost advantages, especially in large codebases.
12Importance of Refactoring
AI-assisted refactoring has the potential to improve code quality while reducing costs. This experiment could redefine the role of refactoring in future software development processes.
Original article: Martin Fowler — Yapay Zeka Destekli Kod Refactoring'in Ekonomik Faydaları