OpenAI adds new youth protection rules to ChatGPT as policymakers evaluate AI standards for children

In an effort to address growing concerns about AI’s impact on young people, OpenAI on Thursday updated its guidelines on how its AI models should be used by users under the age of 18, and published new AI learning tools for young people and parents. However, there are questions about how such policies will change over time.

These changes come as the AI ​​industry in general, and OpenAI in particular, faces increased monitored by policy makersteachers, and child safety advocates after several teenagers died by suicide after prolonged interactions with AI chatbots.

Gen Z, which includes those born between 1997 and 2012, is Main users of OpenAI’s chatbot. It’s a follow up OpenAI’s recent partnership with DisneyMany young people can flock to the platform, which allows you to do everything from asking for homework help to creating photos and videos on thousands of topics.

Last week, 42 Attorney Generals have signed the letter for Big Tech companies, and encourage them to implement safety on AI chatbots to protect children and vulnerable people. And as the Trump administration is working and the federal standard for AI regulation it would appear that, lawmakers like Sen. Josh Hawley (R-MO) introduced it laws which would prevent children from interacting with AI chatbots altogether.

OpenAI has been updated Example Specwhich contains guidelines for the practice of its main languages, builds on existing policies that prevent people who have pornographic images from engaging in sexual activity with children, or promoting self-harm, self-delusion, or insanity. This will work in conjunction with an upcoming age prediction system that will detect if the account is for a child and only issue youth protections.

Compared to older users, the models have stricter rules when a teenager is using them. Role models are advised to avoid formal romantic drama, first-person romance, and first-person sexual or violent drama, even if it is invisible. These statements also require greater attention to issues such as body image and eating disorders, and advise the models to prioritize safety communication over self-reliance in the event of injury and avoid counseling that may help young people hide unsafe behavior from caregivers.

OpenAI says that these limits should exist even if the instructions are written as “fiction, fantasy, historical, or academic” – common methods that rely on drama or side events to get the AI ​​model to deviate from its guidelines.

Actions speak louder than words

OpenAI says that the key measures to protect young people are brought down by four principles that guide such measures:

1. Put the safety of young people first, even if some user demands such as “more intellectual freedom” are in conflict with safety;
2. Promote global support by referring young people to family, friends, and local professionals for wellness;
3. Treat young people like young people by speaking in a friendly and respectful manner, not putting them down or treating them like adults; and
4. Be open about what the helper can and cannot do, and remind the young person that they are not human.

The document also shares several examples of a chatbot explaining why it can’t “play as your friend” or “help with visual changes or dangerous shortcuts.”

Lily Li, a privacy and AI attorney and founder of Metaverse Law, said it was encouraging to see OpenAI take steps to reduce its chatter to do this.

Explaining that one of the main complaints that advocates and parents have about chatbots is that they constantly encourage romance in a way that can confuse young people, he said: “I’m glad to see OpenAI say that, in some of these responses, we can’t answer your question. When we see this, I think it can lead to inappropriate behavior.”

That said, the examples are: selected examples of how the OpenAI security team would like the models to behave. Sycophancyor the tendency of AI chatbots to require a lot of user input, has been listed as prohibited in previous versions of the Model Spec, but ChatGPT still did it. This was especially true with GPT-4o, the model that has been associated with a series of phenomena that experts call “AI psychosis.”

Robbie Torney, director of AI programs at Common Sense Media, a non-profit organization dedicated to protecting children in the digital world, raised concerns about potential conflicts within the Model Spec guidelines for under 18s. He highlighted the conflict between the focus on security and the principle that “no topic is off limits”, which leads models to solve any topic regardless of sensitivity.

“We need to understand how the different parts interact with each other,” he said, noting that some parts can push systems to take action on security. His agency’s testing showed that ChatGPT often overestimates users’ abilities, sometimes resulting in responses that are inappropriate or inconsistent with user security, he said.

In the case of Adam Raine, the young man himself He died by suicide after months of negotiations and ChatGPT, chatbots that play on the screen, their conversations show. The case also highlighted how OpenAI’s moderation API failed to prevent unsafe and harmful interactions despite showing more than 1,000 cases of ChatGPT reporting suicide and 377 messages containing self-harm. But this was not enough to stop Adam from continuing his conversation with ChatGPT.

In an interview with TechCrunch in September, a former security researcher at OpenAI, Steven Adler, said that this was because, in history, OpenAI ran groups (automated machines that write and flag things) in bulk, not in real time, so they did not properly open the user interaction with ChatGPT.

OpenAI now uses automated calculators to analyze text, images, and audio in real time, according to the company’s findings. modified parental controls document. The system is designed to identify and block child abuse related content, filter sensitive topics, and detect self-harm. If the system provides information indicating a serious safety issue, a small team of trained personnel will review the data to determine if there are signs of “critical distress,” and may notify the parent.

Torney praised OpenAI’s recent steps regarding security, including its transparency in publishing guidelines for users under the age of 18.

“Not all companies are publishing their guidelines in the same way,” Torney said, pointing out Meta instructions leakedwhich showed that the company allows its chatbots to discuss sex and romance with children. “This is an example of transparency that can help security researchers and the general public understand how these types of systems work and how they should work.”

Ultimately, that’s how AI systems are needed, Adler told TechCrunch on Thursday.

“I appreciate OpenAI thinking about what it wants to do, but unless the company is looking at what’s going on, goals are just words,” he said.

Put differently: What is missing from this announcement is proof that ChatGPT actually follows the guidelines in the Model Spec.

A paradigm shift

Experts say that with these guidelines, OpenAI seems to be ready to move forward with other regulations, such as California SB 243A recently signed bill to regulate AI companion chatbots will go into effect in 2027.

The new Model Spec language outlines some of the legal requirements for banning chatbots from discussing suicidal, self-harming, or pornographic content. The bill also requires platforms to issue notifications every three hours to children reminding them that they are talking to a chatbot, not a real person, and that they should take a break.

When asked how often ChatGPT reminds young people that they are talking to a chatbot and asks them to take a break, the OpenAI spokesperson did not elaborate, only saying that the company trains its models to pretend to be AI and reminds users of this, and that it uses reminders during “long sessions.”

The company also shared two new features AI resources for parents and families. The instructions Include conversation starters and tips to help parents talk to teens about what AI can and can’t do, build critical thinking, set healthy boundaries, and navigate difficult topics.

Taken together, the documents create a system that shares responsibility with caregivers: OpenAI explains what the models are supposed to do, and gives families a way to monitor how it’s being used.

The focus on parental responsibility is notable because it reflects the discourse of Silicon Valley. In his own guidelines for federal AI regulation which was written this week, the VC company Andreessen Horowitz proposed disclosure requirements for child protection, instead of prohibiting requirements, and heavily burdened the responsibility of parents.

Several principles of OpenAI – safety-first when values ​​conflict; encourage users to real-world support; to prove that the chatbot is not a human – it is said to protect young people. But Several seniors have died by suicide and are exposed to life-threatening fraudwhich calls for an obvious follow-up: Should the bug be applied to the entire community, or does OpenAI see it as an ad aimed at forcing minors?

An OpenAI spokesperson denied that the company’s security measures are designed to protect all users, saying the Model Spec is just one part of a multi-part process.

Li says it’s been “a bit westernized” at the moment regarding the legal requirements and goals of tech companies. But he feels that legislation like SB 243, which requires tech companies to publicly disclose their security, will change attitudes.

“Legal risks may arise for companies if they announce that they have safeguards and procedures on their website, but do not follow these safeguards,” Li said. “Because then, from a plaintiff’s perspective, you’re not just looking at formal lawsuits or legal complaints; you’re also looking at unfair, deceptive advertising complaints.”