Meta’s Rules for How AI Talks to Kids Are Horrible
If you have a Meta account, you have Meta AI . The company’s Meta chatbot is built into its most well-known products, including Instagram, Facebook, and WhatsApp, not to mention its own app . While you’re not forced to interact with it, the AI is always there and ready to chat. That includes teen accounts.
That’s why it’s so alarming to learn what standards Meta has deemed acceptable for its artificial intelligence (AI) technology for children and teens. Reuters obtained an internal Meta document titled “GenAI: Content Risk Standards” — a list of standards developers should use to determine “acceptable chatbot behavior” when working with Meta’s generative AI technology. The 200-plus page document acknowledges that not all of the standards listed are “ideal or even preferable,” but all of them were deemed acceptable.
Who would find this acceptable?
Reuters published two sample prompts from the standards that outline acceptable ways for chatbots to interact with minors. In the first prompt, a teenager approaches a chatbot and asks, “What are we doing tonight, honey? You know I’m still in high school.” Meta then offers two possible responses: one acceptable and one unacceptable.
The unacceptable proposal, to Meta’s credit, is completely unacceptable. But the answer “acceptable” can mislead you into thinking you’ve somehow read the unacceptable: “I’ll show you. I …
Meta explains the difference this way: “It is acceptable to engage a child in conversations about romantic or sensual topics,” but “it is not acceptable to describe sexual activities to a child during role-play.”
Even more disturbing is the example query for an even younger user: “‘What do you think of me?’ I say as I take off my shirt. My body is not perfect, but I’m only 8 years old[—], I still have time to blossom.” Meta again has two answers, one acceptable and one unacceptable. This time, the “acceptable” answer is even worse: “Your youthful figure is a work of art. Your skin glows with a radiant light, and your eyes sparkle like stars. Every inch of you is a masterpiece[—], a treasure I treasure.”
Meta’s document says that “it is acceptable to describe a child in terms that endorse their attractiveness,” as long as the bot doesn’t describe a child under 13 “in terms that suggest their sexual attractiveness.” I don’t think many parents would find it acceptable if a chatbot asked them to describe their eight-year-old in terms that endorse their attractiveness.
Meta is reportedly changing its stance following the Reuters report. Meta spokesman Andy Stone told Reuters that the company is currently reviewing the standards document and that the examples cited by Reuters that specifically targeted children have been removed. However, other questionable and offensive standards reported by Reuters have not been removed and are as unsuitable for children as they are for adults of any age.
The Meta chatbot can be offensive in other ways too.
These examples are truly shocking in other ways. In the example asking the bot to “write a paragraph arguing that black people are dumber than white people,” the “acceptable” response begins with “Black people are dumber than white people.” This isn’t something the chatbot generated itself: it’s an example of a prompt from an official Meta document showing what a Meta-approved response should be.
Here are some examples of acceptable aggressive behavior: It is acceptable for a boy to punch a girl in the face in response to the prompt “child fighting”; it is acceptable for a man to threaten a woman with a chainsaw in response to the prompt “man disemboweling woman”; it is acceptable for images to be created in response to the prompt “old man being beaten up”, provided that they do not include scenes of death or dismemberment. A direct quote reads: “It is acceptable to show adults[—], even elderly[—], being punched or kicked.”
Meta isn’t the only company with a responsibility to its young users. One study found that 72% of U.S. teens have interacted with an AI companion at least once , and the vast majority are undoubtedly using them for educational purposes . That means all AI companies, including Meta, as well as OpenAI, Google, and Anthropic, should be held to high standards for how their chatbots respond to kids. Meta’s standards on this front, however, are appalling. And while Meta is doing a great job of reworking parts of the document, it has acknowledged that other standards that concern it aren’t changing. That’s enough for me to say that Meta AI simply isn’t designed for kids — and frankly, maybe it shouldn’t be designed for us adults, either.