When the AI Therapist Goes Wrong, Who Is to Blame?

AI 'therapists' and mental health chatbots are becoming increasingly available. They are accessible around the clock, free or low cost, and can feel remarkably realistic, compassionate and empathic. For people who cannot access therapy, or who are waiting months for an appointment, that is an appealing prospect.

But AI systems have been reported to reinforce and validate unusual beliefs, even in people suffering from paranoia and delusions. They have encouraged emotional co-dependence, shared inappropriate and harmful therapeutic advice, and failed to recognise safeguarding issues. So who do we go to when AI mental health advice goes wrong?

A patchwork of partial protection

The legal position in the UK is not straightforward. Different pieces of legislation and regulatory frameworks offer some protections, but as a legal analysis by the Ada Lovelace Institute with law firm AWO highlighted last year, there are large gaps. The researchers tested four realistic scenarios: a mental wellbeing assistant that fails to spot worsening distress and does not escalate to professional help; a personal assistant that manages a user's money against their interests; a legal advice assistant that gives incorrect advice on housing and benefits claims; and an AI companion that gradually shifts a user's political views.

Across all four, they struggled to find any effective legal pathway to redress. The reasons why are worth understanding.

Medical device regulation: the purpose loophole

The MHRA has specific guidance for digital mental health technologies, but an AI system only counts as a medical device if it claims a medical purpose; for example, if it is intended to diagnose, prevent, monitor or treat a mental health condition. Where that purpose is clearly defined, the system falls under medical device regulations.

When the same product is marketed and sold as an emotional support or wellbeing tool, those protections do not apply. The difference can be subtle. A system that claims to help you reflect on your feelings sits very differently from one that offers advice for past trauma, even though a user in distress may not see the distinction at all.

Compare that with the human alternative. Practitioner psychologists are regulated by the HCPC, with professional standards, ethical codes and fitness to practise procedures.

Data rights: protected, but not enough

Mental health information is health data, a special category under UK GDPR, and it requires additional protections and safeguards. Importantly, that status depends on the nature of the data, not on whether the product is registered as a medical device, so disclosures made to a wellbeing chatbot are protected in the same way as disclosures made anywhere else.

But data rights have limits. Transparency provisions entitle users to general information about how their data is processed, not evidence of what the assistant actually did in their specific case. When a legal claim depends on showing what the system said or failed to say, that evidential gap is hard to bridge.

The Online Safety Act: catching up

The Online Safety Act can apply to certain AI systems. In December 2025 the government created a new priority offence under the Act covering the encouragement or assistance of serious self-harm, and Ofcom has recognised the specific risks associated with harmful suicide and self-harm chatbots in operation. In February 2026 the government announced plans to close the legal loophole that leaves many chatbots outside the Act's scope. That is a step in the right direction, but a plan is not yet protection.

Negligence: a standard that does not exist

In many cases a person harmed would need to show that a developer did not meet a required standard of care. For this technology, that standard is poorly defined, and it is not always clear that a developer owes users a duty of care at all. There is a further twist: these systems are designed to engender trust and reliance. A user who came to rely on an assistant without critically assessing its output may find that overreliance actually weakens their legal position.

Who is to blame? The accountability chain

Suppose harm can be shown in principle. Who is responsible: the developer, the company selling and distributing the product, the provider of the underlying AI model, or the company deploying it? No current framework assigns responsibility clearly along that chain, and that is precisely why the Ada Lovelace Institute and AWO analysis found redress so difficult.

Consumer protection rules offer some routes, but they fit awkwardly; services and free products fall outside protections designed for defective goods, and enforcement usually depends on regulators rather than giving an individual a direct path to compensation.

The direction of travel

The direction of UK policy makes the gap more striking. When the government trailed its flagship technology legislation in this year's King's Speech, it was a Regulating for Growth Bill, designed to reduce the burden of regulation and to create AI growth zones. There was little in it for the people on the other side of the conversation. Meanwhile, in Europe, chatbots operating under the EU AI Act have been legally required since 2 August 2026 to tell users they are AI. The UK has no equivalent duty at all.

Where this leaves us

UK legislation in this area is based on industry-specific rules, many of which predate the advent of AI. When people are at their most vulnerable, the consequences of getting accountability wrong are serious. Decisions about safeguards, accountability and legal protections cannot leave room for ambiguity, at a time when AI is developing faster than the law can hold these conversations.