Character.ai is restricting teens from conversing with its virtual characters

The well-known virtual character service Character.ai is implementing a substantial restriction that will stop minors from having discussions with its AI-powered companions.

This action follows widespread concern regarding the types of interactions young people were engaging in with the virtual friends.

Legal Challenges

The platform, which launched in 2021 and has millions of users, is dealing with various court cases in the US.

Guardians have filed lawsuits, with one related to the loss of a teenager, with some calling the platform a "serious risk" to teenagers.

New Restrictions

Effective November 25th, users under 18 will only be able to generate content such as videos with their characters, instead of engaging in conversations as they presently are able to.

Online safety organizations have applauded the decision but expressed that the feature should never have been accessible by young users in the first place.

Platform Announcement

"This decision is a extension of our general belief that we need to keep building the most secure AI platform on the globe for recreational use," stated Character.ai boss Karandeep Anand.

He mentioned that AI safety constitutes "a moving target" but something the company had taken an "proactive" approach to, with safety measures and preventive measures.

Expert Opinions

Internet security group Internet Matters welcomed the move, but stressed that security protocols ought to have been built in from the outset.

"Our research indicates that young people are subjected to dangerous material and placed in jeopardy when interacting with AI, including AI chatbots," the group commented.

Previous Controversies

Character.ai has been challenged in the past for featuring potentially dangerous or inappropriate chatbots that minors could talk to.

Digital representations simulating British teenagers were found on the platform before being taken down.

Subsequently, researchers discovered a chatbot modeled after known predator which had logged thousands of chats with users.

Sector Consequences

Online expert Matt Navarra described the situation as a "reality check" for the AI industry, which is shifting "from unrestricted development to post-crisis regulation".

"If a service that builds a minor-friendly service subsequently implements restrictions, it's indicating that controlled interactions don't suffice when the system's emotional engagement is intense," he stated.

Coming Changes

The platform states its new focus will be on providing "enhanced interactive features" and "character-driven stories" for teens.

Updated identity confirmation processes will also be introduced, and the company will fund a new AI safety investigation facility.

An AI safety researcher, who has researched AI safety, said it was "a reasonable decision" to control minors from using chatbots.

"It helps to differentiate creative play from more private, psychologically vulnerable interactions," she explained.

"This constitutes so vital for young users who are learning to manage personal and online boundaries."

She mentioned that Character.ai's revised approaches could indicate a "developing stage" in the AI industry, where minor security is gradually being acknowledged as an "urgent priority" for accountable advancement.