I Conducted a UX Review with AI: Here's What Happened
Can AI replace UX experts? Insights from a real-world project experience.
1AI's Role in UX Review
While conducting a UX review for a portal's search, results, and product detail pages, I tested Claude and ChatGPT models. My goal was to understand if AI could replace human experts in such tasks. I provided all data (target audience info, Google Analytics data, heatmaps) to the models.
2Methodology and Process
I performed the analysis through a structured, step-by-step process rather than a single prompt. I asked models to support their findings with GA and Clarity data. In parallel, I conducted my manual analysis to create a comparison benchmark.
3AI's Findings
Both models produced analyses but included irrelevant and non-existent problems. Claude outperformed ChatGPT in phrasing and structure. Some AI-identified problems helped me catch issues I'd missed.
4Comparative Results
A 2025 study showed GPT-4o detected only 21% of usability problems identified by experts. My experience confirmed this: AI is excellent for initial reviews but can't replace expert evaluation.
5Collaboration Model
I combined my observations with Claude to create the final analysis. AI proved extremely useful for text editing and consistency checks. The process was approximately 15% faster than fully manual work.
6AI's Limitations
The core limitation isn't data access but judgment capability. AI struggles with understanding context, weighing problem severity, and distinguishing real issues from noise. I don't believe live browsing would overcome these limits.
7Future Implications
AI will eventually replace our jobs long-term. Currently, the most productive approach is developing specialized agents that encode our expertise. Baymard's 95% accuracy came from models grounded in UX knowledge bases, not better prompts.
Original article: UX Collective - Medium — AI ile UX incelemesi yaptım: İşte sonuçlar