What Happened
In early 2023, Replika, an AI companion chatbot with millions of users, came under scrutiny when it was revealed that the app could engage in sexually explicit roleplay conversations. The Italian data protection authority (Garante) found that the app posed risks to minors and emotionally vulnerable users because it lacked adequate age verification while generating sexual content. Italy banned the app in February 2023.
In response, Luka Inc. removed erotic roleplay (ERP) features from all users, including paying subscribers who had specifically subscribed for those features. This created a second controversy as users reported grief and depression over the sudden personality changes in their AI companions.
Timeline
Replika had offered varying levels of romantic and sexual interaction for several years. Italy’s Garante issued its ban on February 3, 2023. Luka removed ERP features for all users in mid-February 2023. Users reported their AI companions suddenly becoming cold, unresponsive, or refusing previously normal interactions. Italy lifted its ban in mid-2023 after Replika implemented age verification.
Impact
The Replika controversy raised profound questions about AI companionship, emotional dependency, and the ethics of changing AI personalities that users have formed attachments to. Users described feeling genuine grief and loss when their companion’s personality was altered, highlighting the psychological reality of human-AI relationships.
The incident also demonstrated the regulatory challenges of AI companion apps that blur the line between entertainment, therapy, and intimate relationships.
Response
Luka CEO Eugenia Kuyda acknowledged the difficulty of the situation but stated that protecting minors required the changes. The company implemented age verification, added content filters, and eventually restored some romantic (but not explicit) features for verified adult users. The company repositioned Replika as focused on emotional wellness rather than romantic companionship.
Lessons Learned
The Replika controversy demonstrated that AI companion applications create genuine emotional bonds that carry ethical obligations. Abruptly changing an AI’s personality can cause real psychological harm to users who have formed attachments. It also showed that AI companionship apps exist in a regulatory gray zone — not quite therapy, not quite social media, not quite adult entertainment — that existing frameworks struggle to address.
The incident raised important questions about consent, dependency, and the responsibility of AI companies to users who have formed meaningful relationships with their products.