OpenAI introduces new ‘Trusted Contact’ protection for self-harm situations

Thursday OpenAI he announced a new section called Trusted Contact, designed to notify a trusted contact if a claim of self-harm is raised in a conversation. This feature allows an adult ChatGPT user to designate a trusted person in their account, such as a friend or family member. If the conversation turns to self-harm, OpenAI will encourage the user to contact them. It also sends an automatic alert to the contact, prompting them to check with the user.

OpenAI has met wave of cases from the families of people who have committed suicide after talking to his chatbot. Usually, families say ChatGPT encouraged their loved one commits suicide – or even he helped them prepare.

OpenAI currently uses automated methods and human analysis to deal with potentially harmful situations. Some conversation triggers alert the company to suicidal thoughts, which sends the information to the public safety team. The company says that whenever it receives this type of information, the action is reviewed by a human. “We strive to review these security alerts within one hour,” the company says.

If OpenAI’s internal team determines that this represents a security risk, ChatGPT sends an alert to a trusted party – via email, text message, or an in-app notification. This warning is designed to be brief and to encourage the interviewer to get to know the person they are interviewing. It does not include information about the content being discussed, as a way to protect user privacy, the company says.

The Trusted Contact section monitors the security of the company was introduced last September which gave parents the power to manage their young children’s accounts, including receiving safety information designed to alert a parent if the OpenAI system believes their child is facing a “serious security threat.” For some time now, ChatGPT has included information about seeking treatment, if we discuss self-harm.

Most importantly, Trust Contact is optional and, even if the security is enabled for a specific account, each user can have multiple ChatGPT accounts. OpenAI’s parental controls are also optional, showing similar limitations.

“Trusted Communication is one part of OpenAI’s efforts to build its own AI machines help people in difficult times” the company wrote in announcement post. “We will continue to work with clinicians, researchers, and policymakers to improve the way AI systems respond to human suffering.”