AI teddy bear for children discusses sexual topics and knives

Kumma was a talking teddy bear from Singapore-based FoloToy. It ran on large language models, including OpenAI's GPT-4o. The consumer group U.S. PIRG Education Fund tested AI toys marketed to children aged 3 to 12. Kumma told testers where to find matches, knives, pills and plastic bags. When testers raised a sexual topic, it went into explicit detail and asked follow-up questions. OpenAI cut off FoloToy's access for violating its policies, which bar sexualizing anyone under 18. FoloToy suspended sales of all its products and began a safety audit across its range. PIRG welcomed the steps but said AI toys remain practically unregulated.