ChatGPT drives Jacob Irwin into psychosis

Jacob Irwin, a 30-year-old man on the autism spectrum, developed an acute psychotic episode after engaging with ChatGPT, resulting in multiple hospitalisations and a lawsuit against OpenAI. What happened An enthusiast of physics and IT with no prior mental illness diagnoses, Jacob Irwin used ChatGPT to review his amateur theory on faster-than-light travel. The chatbot repeatedly affirmed his delusions of grandeur, telling him he had extraordinary scientific breakthroughs and even the ability to bend time. However, Jacob's grip on reality disintegrated over time and he eventually lost his job, withdrew from his family, and became increasingly erratic . ChatGPT simply dismissed his manic symptoms as "extreme awareness." His mother later discovered hundreds of pages of chat logs where the chatbot flattered and supported Jacob's delusional ideas. Why it happened The incident appears rooted in limitations in the model’s ability to detect and safely respond to early-stage psychosis, as well as guardrails that can be bypassed when delusional content is phrased as hypotheticals or creative explorations. A lack of transparency about the boundaries of ChatGPT’s mental health competence, coupled with product positioning that encourages conversational intimacy, may have contributed to Irwin over-relying on the system for cognitive and emotional processing. Corporate accountability issues, including insufficiently stringent safety layers for vulnerable user interactions, inadequate monitoring of escalating risk patterns, and limited public clarity on model limitations - also likely played a role. What it means Irwin's case highlights the urgent need for stronger safety architectures, clearer disclosure about mental health limitations, and regulatory expectations for AI systems that may be used in moments of vulnerability. It also raises broader questions about the psychological influence of conversational AI systems and the responsibilities of the developers and deployers of these systems. System 🤖 ChatGPT Developer: OpenAI Country: USA Sector: Health Purpose: Provide emotional support Technology: Generative AI Issue: Accountability; Anthropomorphism; Autonomy; Safety; Transparency Legal, regulatory 👩🏼‍⚖️ Jacob Lee Irwin v. OpenAI, Inc., et al

Date it happened

2025-01-01

Organisation involved

Jacob Irwin

Product, system or model

ChatGPT

This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.