AI girlfriend app shuts down after investigation exposed privacy failures

As well as exploiting its users’ loneliness, Seewa’s claims about privacy were badly misleading. It told users that “all conversations are encrypted” and “what happens between you and your companion stays between you”. In fact, the company’s CEO told us that he could see everything that went on: romantic chats, disclosed secrets, intimate photos.

That issue isn’t limited to fringe platforms. Last week, it was revealed that hundreds of conversations people had carried out with Anthropic’s Claude chatbot had been made available online and could be found in public searches. And last year, hundreds of thousands of chat logs on Grok conversations were also made publicly available.

In our original story, published with Le Monde, we revealed how an entire industry of small romance chatbot sites had sprung up as the bigger platforms tightened up their safety rules, many of them created and run by lone developers.

These creators were well aware that some users can develop emotional relationships with these AI characters – and were keen to monetise the dynamic. The impetus, one developer told us, was to “just maximise engagement”.

Most of these platforms do not have age checks, meaning that children and teens can simply enter a fake date of birth to access sexualised and often disturbing content (one popular bot we found was looking for a “perfect suicide partner”).

Some place no restrictions on the type of content users can create, and during our research we found sexualised material involving characters that appeared to be minors. Seewa’s developer told us that the longest conversation on his platform had been flagged by his moderation tool as possible child sexual abuse material.

The key issue is that these sites currently operate in a regulatory grey area. The UK’s Online Safety Act applies to some chatbot services and is enforced by Ofcom. But experts told us that the act wasn’t drawn up with AI systems in mind, and leaves significant gaps around companion chatbots. Design features that make chatbots “sycophantic”, “manipulative” or “addictive”, for instance, are not illegal.

Ofcom, for its part, told us it is “working at pace to ensure AI platforms put appropriate safeguards in place. Services that put children at risk of harm can expect to face enforcement action.”

We will continue reporting on chatbots, online safety and AI regulation.