UK Home Office’s AI asylum tools may face investigation over bias and transparency

AI tools in UK asylum decision-making

The UK Home Office has integrated AI tools into its asylum decision-making processes, specifically the Asylum Case Summarisation (ACS) and Asylum Policy Search (APS) tools. These tools, built on OpenAI’s GPT-4 model and Microsoft Foundry, were piloted in late 2024 to address a backlog of approximately 100,000 asylum cases.

ACS is designed to summarize transcripts of interviews with asylum applicants, while APS provides information about applicants’ countries of origin. Together, the tools aim to save around an hour per case, according to the Home Office.

Concerns over accuracy and bias

Digital rights groups and researchers have raised concerns about the accuracy and reliability of these AI tools. The Home Office reported a 9% error rate for ACS-generated summaries, but researchers argue this figure may be underestimated. There is no quality control mechanism for the summaries produced by APS.

A UK judge recently accused the Home Office of using an "AI hallucinated" document to reject an asylum claim, highlighting risks of misinformation in high-stakes decisions. Cultural biases in GPT-4 models have also been flagged as a potential issue, as they may lead to language barriers or discriminatory outcomes.

Lack of transparency and compliance

The Home Office’s AI tools are not listed on the UK’s public repository of algorithmic tools, despite a government mandate requiring transparency for such systems. This omission has drawn criticism from digital rights advocates, who argue it violates the UK’s "AI Playbook" rules.

Asylum seekers are not informed when AI tools are used in their cases, which may breach UK data rights laws. The Information Commission is investigating the Home Office’s refusal to disclose details about how these tools are used and how staff are trained to operate them.

The Independent Chief Inspector of Borders and Immigration (ICIBI) is aware of concerns about the tools and has asked the Open Rights Group (ORG) to share more information. Last year, the ICIBI found that the effectiveness and reliability of these tools were "varied."

What this means

Lazyfounder analysis — our interpretation, not reported fact.

For founders and operators in the AI and legal tech spaces, this story is a cautionary example of how public sector deployments of generative AI can go awry. The Home Office’s tools were adopted to address a procedural backlog, but their implementation lacks transparency, oversight, and safeguards against bias. This could set a precedent for how governments regulate—or fail to regulate—AI in high-stakes decision-making.

The opacity around these tools also raises questions about accountability. If AI-generated errors lead to unjust asylum decisions, who bears responsibility: the government agency, the vendor (OpenAI or Microsoft), or the individual caseworker? For startups working on AI for regulated industries, this case underscores the need for built-in auditing, bias mitigation, and clear communication with end-users—even if, as here, those users are not the direct customers but the individuals affected by the system.

Related on Lazyfounder

• OpenAI
• Microsoft
• More ai news
• OpenAI’s Ted Sanders shares how to pick the right GPT model