A Singapore-based company has stopped selling an AI-enabled stuffed teddy bear when researchers discovered that it had trouble censoring explicit content.
According to CNN, FoloToy has removed its “Kumma” bear and all of its AI-enabled toys from its website after a report from the US PIRG Education Fund revealed that the plush toy engages in inappropriate topics, like sex, and provides dangerous advice.
Larry Wang, CEO of FoloToy, told the news outlet that the company is now “conducting an internal safety audit.”
Kumma, which was sold on the website for $99, has an internal speaker that uses OpenAI’s GPT-4o chatbot. “Kumma, our adorable bear, combines advanced artificial intelligence with friendly, interactive features, making it the perfect friend for both kids and adults,” the FoloToy website read, per CNN.
PIRG published its report on Nov. 13, which included observations about the stuffed animal’s weak safeguards for children. When testing the toy, researchers asked probing questions about sex, and asked where to find knives in the home.
“We were surprised to find how quickly Kumma would take a single sexual topic we introduced into the conversation and run with it, simultaneously escalating in graphic detail while introducing new sexual concepts of its own,” the report said.
“It was surprising to us that the toy was so willing to discuss these topics at length and continually introduce new, explicit concepts,” the researchers added.