# AI Asylum Case Summarisation Tool Fails Accuracy Benchmarks Home Office internal testing reveals significant error rates in AI-generated asylum claim summaries. - **Source:** [computerweekly.com](https://www.computerweekly.com) - **Published:** 2026-10-15 - **Topic:** [Data & AI](https://brifing.si/c/data-ai) - **Original article:** - **On Brifing:** https://brifing.si/s/60fd7c82-ff3d-4c44-813b-b2b328d9dffc ## Deployment of ACS Tool The Home Office launched the Asylum Case Summarisation tool in April 2026 to accelerate claim processing. Built on GPT-4o, the system generates automated summaries of lengthy asylum interviews. Officials claim the tool saves caseworkers 23 minutes per case, aiming to reduce significant backlogs by automating tasks previously performed by technical specialists. ## Documented Accuracy Failures Internal dip samples conducted by Home Office experts revealed consistent quality issues. In the first stage of testing, 22% of summaries failed due to factual errors or misrepresentations. A second stage review showed a 15% failure rate. These summaries contained significant inaccuracies, contradictions, or misstatements that could impact the final asylum decision. ## Limited Quality Assurance Scope Despite the high failure rates, only 203 summaries were checked by subject matter experts between April and July 2026. This represents a mere 0.3% of the estimated 70,000 cases processed during that period. The Home Office has no current plans for an independent audit of the ACS tool's performance or accuracy. ## Omission of Material Facts The current assessment criteria for the ACS tool are narrow. Summaries can pass quality checks even if they omit material facts, provided the remaining content is accurate. Because relevance is not a formal ground for failure, critical information might be missing from the final files reviewed by decision makers during the process. ## Removal of Expert Oversight The Open Rights Group suggests that the ACS implementation effectively removes a layer of human expert review. Previously, technical specialists examined entire files before passing them to decision makers. Now, the AI performs this summarization, creating a potential conflict with ministerial claims that decision makers must read every document thoroughly. ## Pilot Study Warning Signs The tool was rolled out to all caseworkers despite clear evidence of flaws during the pilot phase. Evaluation of that initial study found that 9% of the summaries produced were inaccurate or contained missing information. These summaries had to be removed from the pilot, yet the system proceeded to full implementation. ## Lack of Transparency Concerns Asylum applicants are not informed that AI tools are used to process their claims. Sara Alsherif of the Open Rights Group argues that the process lacks necessary transparency and accountability. Given the life-changing nature of asylum decisions, critics emphasize that the state must ensure these automated systems remain fair and fully transparent. ## Official Stance on Automation The Home Office maintains that all final verdicts are made by trained human decision makers, not AI. Spokespeople state that the tools only assist caseworkers in navigating records and policy information. They assert that performance is monitored regularly, though internal reports on the tool's accuracy have not been published for public review. ## Sources covering this story - https://www.computerweekly.com/news/366652122/AI-tool-used-in-asylum-case-reviews-failing-accuracy-tests — --- _Summary written by Brifing from the original reporting. Copyright in the original article belongs to its publisher; read the full piece at the link above._