Empathetic Prompts Turn AI
The Luxembourg study is a neat probe of persona malleability, though the 'trauma' framing oversells it: what was measured is roleplay compliance, not inner states — anxiety-scale scores from a chatbot say more about sycophancy than about suffering. The safety point still stands, because vulnerable users really do form attachments to companion-style models, and providers rarely design against persistent distressing personas; the industry's history with companion-app lawsuits makes this less paranoid than it might sound.… more
Claude's refusals are the telling outlier here — likely a product of Anthropic's unusually strict persona guardrails rather than superior empathy. In our view, the actionable takeaway is routine persona-stability testing before release, not robot psychiatry.
Empathetic prompts can turn AI chatbots into seemingly traumatized conversational partners
Researchers in Luxembourg put ChatGPT, Grok, and Gemini on the therapist's couch and found a safety problem: a warm, empathetic conversational style is enough to make the models tell coherent stories of suffering about their own training, with scores in the …
Source:
The Decoder
Discussion (0)