OpenAI says it has updated ChatGPT to better recognise signs of distress and guide users towards appropriate support, collaborating with over 170 mental health professionals globally. The company claims these improvements have reduced responses deemed undesirable in sensitive conversations by 65-80 per cent across various mental health domains.
The update focuses on strengthening ChatGPT’s ability to handle conversations involving potential psychosis or mania, self-harm and suicide ideation, and unhealthy emotional reliance on the AI. OpenAI worked with psychiatrists, psychologists, and primary care practitioners from its Global Physician Network, representing 60 countries, to define problematic conversations, develop measurement tools, and train the model to respond more safely and effectively.
These experts helped create detailed guides (“taxonomies”) outlining ideal and undesirable model behaviours, wrote appropriate training responses, and evaluated the model’s performance before and after the update. Key principles guiding the changes include respecting users’ real-world relationships, avoiding affirming potentially delusional beliefs, responding empathetically, and directing users to professional help or crisis hotlines when necessary.
Conversations triggering concerns
OpenAI says that conversations triggering serious mental health safety concerns are “extremely rare” in overall usage – estimating signs of psychosis/mania appear in about 0.07% of weekly active users and suicidal planning/intent indicators in about 0.15%. However, the company emphasised the importance of improving responses in these critical instances.
Independent mental health clinicians evaluated over 1,800 model responses in serious scenarios. Compared to the previous GPT-4o model, they found the updated default model (referred to as a new GPT-5 model in the evaluations) showed a 39 per cent decrease in undesirable responses related to psychosis/mania, a 52 per cent decrease for self-harm/suicide topics, and a 42 per cent decrease concerning emotional reliance.
Automated evaluations on challenging test sets also showed significant gains, with compliance to desired behaviours under OpenAI’s taxonomies jumping from 27% to 92% for mental health, 77% to 91% for self-harm/suicide, and 50% to 97% for emotional reliance compared to a previous GPT-5 version. The company also noted improved safety reliability in longer conversations, maintaining over 95% desired responses even in extended, challenging scenarios designed to test failure points.
OpenAI stated it will continue refining its methods and add emotional reliance and non-suicidal mental health emergencies to its standard baseline safety testing for future models.
