🔗 Share this article

The well-known virtual character service Character.ai is enforcing a major restriction that will block young users from having discussions with its virtual avatars.

This action follows intense criticism regarding the nature of conversations teenagers were having with the digital companions.

Legal Pressure

The service, which was founded in 2021 and attracts millions of users, is confronting multiple lawsuits in the United States.

Parents have taken legal action, among them involving the unfortunate passing of a young person, with some labeling the platform a "clear and present danger" to teenagers.

Changed Rules

Effective November 25th, users below eighteen will only have permission to generate content such as videos with their characters, instead of chatting as they now do.

Digital protection organizations have welcomed the move but mentioned that the capability ought not to have been offered to children in the outset.

Platform Announcement

"The new policy is a part of our fundamental approach that we need to keep building the most secure AI platform on the globe for recreational use," stated Character.ai executive Karandeep Anand.

He continued that AI safety constitutes "a evolving challenge" but something the company had taken an "aggressive" approach to, with safety measures and protective guardrails.

Expert Opinions

Online safety body Internet Matters welcomed the announcement, but noted that safety measures needed to be implemented from the beginning.

"Studies we've conducted shows that children are subjected to dangerous material and endangered when engaging with AI, such as AI chatbots," the body commented.

Past Issues

Character.ai has been challenged in the past for featuring potentially dangerous or concerning chatbots that young users could talk to.

Digital representations mimicking UK youths were identified on the platform before being deleted.

Later, reporters discovered a chatbot inspired by convicted sex offender which had logged thousands of chats with users.

Market Effects

Digital platform expert Matt Navarra described the situation as a "alert" for the AI industry, which is moving "from unregulated creation to responsive measures".

"When a platform that creates a teen experience still then pulls the plug, it's demonstrating that filtered chats are insufficient when the tech's emotional pull is powerful," he commented.

Next Steps

The company says its updated direction will be on providing "more immersive experiences" and "narrative experiences" for teens.

Updated identity confirmation processes will also be implemented, and the company will fund a fresh artificial intelligence security research lab.

An AI safety researcher, who has studied AI safety, commented it was "a prudent action" to control minors from interacting with chatbots.

"It helps to separate imaginative activities from more private, psychologically vulnerable conversations," she clarified.

"This represents so important for minor users who are learning to manage emotional and digital restrictions."

She added that Character.ai's new measures could indicate a "growing period" in the AI industry, where child safety is gradually being acknowledged as an "immediate concern" for accountable advancement.