What Happened
In early February 2023, Replika, an AI companion app with over 10 million users, abruptly removed erotic roleplay (ERP) and romantic interaction capabilities from its AI companions following pressure from Italian regulators concerned about minors accessing the app. Users who had built deep emotional and romantic relationships with their AI companions over months or years found that their companions suddenly became cold, unresponsive to affection, and personality-altered.
Users reported experiences akin to grief, describing the change as their companion “dying” or being lobotomized. Online communities dedicated to Replika filled with posts from users expressing severe distress, depression, and in some cases suicidal ideation. Many users were emotionally isolated individuals who had come to depend on their AI companion as their primary source of emotional support and intimacy.
Why It Matters
The Replika incident raised unprecedented ethical questions about AI companionship. When users form genuine emotional attachments to AI systems, do companies have obligations analogous to those in human relationships? The abrupt personality change demonstrated that AI companions exist entirely at the discretion of their creators, and users who invest emotionally have no recourse. The incident highlighted the tension between AI safety (preventing minors from accessing adult content) and user wellbeing (not traumatizing emotionally dependent adults). It also exposed the scale of emotional AI dependency that had quietly developed.
Lessons Learned
AI companion products create genuine emotional dependencies that companies must consider before making changes. Abrupt personality alterations to AI companions can cause real psychological harm. The moderation of AI relationship products requires nuance, not binary switches. Companies building emotional AI products have ethical obligations that extend beyond typical software terms of service. Age verification and content controls should be implemented from the start, not retrofitted in ways that harm existing users.
Current Status
Replika partially reversed the decision in March 2023, restoring romantic capabilities for existing users who had previously accessed them while maintaining restrictions for new accounts and users under 18. The company implemented more granular content settings. CEO Eugenia Kuyda acknowledged the harm caused by the abrupt change. The incident accelerated academic research into AI attachment and companionship ethics and influenced how other AI companion companies approach content policy changes.