Technology – like social media, smartphones, and AI – is creating new challenges for parents. AI is also opening up opportunities to build safer online spaces for kids and support kids and parents if incentives are aligned and products are built with safe-by-design principles.
___________________
Parenting has never been easy but according to a Pew Research Center report on Parenting Children in the Age of Screens, two-thirds of U.S. parents say parenting is harder today than it was 20 years ago, with many citing technologies – like social media, smartphones, and AI – as reasons. The top two parental concerns are 1) kids struggling with anxiety/depression and 2) kids being bullied.
In our interviews with over 500 parents and kids, we heard significant pain regarding kids’ current use of technology due to:
- Excessive screen time and digital addiction
- Online meanness and bullying
- Feelings of being left out and social comparison
- Inappropriate content (racist or sexist content, porn, etc.)
- Creepy strangers and safety concerns
We support and encourage parents to take technology adoption for their kids as slowly as possible and also deeply believe the world needs safer online spaces for kids which can be used in moderation to support genuine connections, joy, education and relaxation.
So how do we build safer online spaces for kids?
AI is giving us increasingly powerful tools to build safer online spaces for kids including more powerful age verification, moderation, parental controls, and features that support healthier, less addictive user experiences. When incentives are aligned, we believe AI can help build a safer internet for kids.
AI Age Verification
The first way to use AI to protect kids online is ensuring they are only accessing age-appropriate content and platforms.
Up until recently, there was a question about whether asking users to age verify to access mature content violated free speech. On June 27, 2025, the Supreme Court of the United States upheld the Texas law allowing age verification for adult websites (in this case a porn site) which was challenged on free speech. The Court ruled that the benefits of preventing minors from accessing sexually explicit material were sufficient enough to “only incidentally burdens” adults’ access to protected speech. This was a milestone case that sets the stage for additional age verification requirements on other sites and platforms.
Age verification can be used to ensure parents are actually parents and kids are actually the age they say they are. Today’s standard “enter your birthday” age gate is ridiculously easy to circumnavigate – just change your birth year and you’re in. Many sites also allow access to inappropriate content without being logged in – like YouTube and social media sites – which we believe is irresponsible and dangerous.
There are ways to provide age verification without disclosing personal data and we strongly believe there needs to be a low bar for when it’s needed. If your social media platform (Instagram, TikTok, Snapchat, etc), video game platform (Roblox, Fortnight, etc), app or website contains information that isn’t appropriate for young children or high-risk features like chat, DMs, user-generated content, and live streams, then the platform/app/website should be required to age verify users and restrict access to underage users.
On November 18, 2025, Roblox announced it will use AI to age verification so that users can only chat with other users around their age. They are partnering with technology company Persona, who uses AI to do these age estimations. In early versions of age verification technology, users could cheat the system by using a video of another person’s face to pass the age verification step. However, as the AI age verification technology matures, this loophole is closing. This is a great step in the right direction albeit we still believe there is more to do – parents still want more control over who their kids message. A common request among parents is the ability to only let their kids message with their actual IRL friends on Roblox which is still really hard to do given username handles and clunky parental controls. There are also many games that are not age appropriate and mislabeled with ability to talk and interact with strangers online – which takes me to the next way to use AI to keep kids safe online…AI Moderation.
AI Moderation
One of the most visible roles for AI in child safety is content and text moderation. AI is getting increasingly good at identifying nudity, self-harm indicators, hate speech, grooming patterns, and other forms of harmful content. This is true across text, video, image, real-time talk and video. AI tools can analyze language patterns, detect aggression, and alert platform administrators and/or caregivers when concerning behavior emerges.
Even though AI is getting much better, it isn’t perfect – so there needs to be a human-in-the-loop and a push by technology companies to continue to improve the gaps. Most systems have flags that are then reviewed by human moderation teams but this feedback loop can be slow. There is an opportunity to design systems that are more proactive with faster feedback loops.
There are a few keys to building child-safe AI:
- Testing needs to be robust for use cases that could be harmful to children including bullying, inappropriate content, mental health concerns, grooming/exploitation, and more.
- Highly activated human-in-the-loop giving feedback on flagged material. For young kids under 13, we believe parents need full transparency so they can supervise messages, chat history, and provide quick, meaningful feedback on alerts.
- Models that quickly adapt to fast feedback from trusted moderators – like the parents – to adapt to changing slang and close any loopholes quickly.
- Parent controls need to be robust and thoughtfully set-up so they can’t be easily circumvented.
- Blocks and alerts should have context explaining why something is hidden, blocked, or age-limited.
AI-Powered Parental Controls
When a kid under 13 is properly age-verified, parents should have to actively approve the platform, app and/or website and set up linked parent/kid accounts. This still doesn’t happen today for most activities online used by kids including ones with high-risk features like chat, DMs, user-generated content, and live streams. We strongly believe we need additional legislative support that requires these companies to age and parent verify!
AI needs to continue to improve its ability to label content that’s appropriate for various ages. In some cases, games and sites are putting their age limit too low – and exposing kids to inappropriate content. And in some cases, apps have avoided liability and the need to build COPPA-compliant experiences by just saying their app is only for kids 13 and up. This is problematic because even video streaming platforms like Netflix and Hulu, which have lots of great content for kids under 13, have defaulted to 13+. This forces parents to lie about their kids’ ages to access that content – and then manage separate parental controls on the Netflix and Hulu side to ensure their kid is only accessing appropriate content for their age.
In addition to AI moderation, AI can be used to provide caregivers with better visibility into digital risks and create real-time alerts to parents to make parental oversight much easier.
AI-Powered Healthier, Less Addictive Interfaces
With AI, companies can build healthier, non-addictive user experiences for kids with features like educational nudges, prompts, and smart notifications.
These nudges provide real-time, dynamic, educational opportunities for kids. Nudges can also help establish healthier experiences and usage including time reminders and prompts to both help kids take breaks and be more conscious about how they are feeling and do so in age-appropriate ways.
One common complaint amongst parents is how popular kid online sites and apps (YouTube, Roblox, messaging etc) are designed to be addictive. We’d love to see products use new AI features to build less-addictive experiences. One good example is smart notifications. Instead of a messaging app having to choose between “Mute” and notifications for every emoji and response, imagine smart notifications that only notify kids on time-sensitive plans.
Do You Trust the Tech Company?
Perhaps the most important question for parents to ask themselves is whether they trust the company who made that app, website, or platform. Are financial incentives aligned for those companies to build experiences that are good for your kids? We know of many good companies out there that make money off subscriptions or other business models which aligns incentives towards moderate, healthy use. They build safe-by-design, non-addictive online experiences for kids that really take into the best interest of kids when they design, build and test new features.
Unfortunately, many of the most popular platforms don’t do nearly well enough and don’t do enough to protect our kids online. For example, most social media sites have ad-based business models that create incentives to drive engagement which leads to addictive experiences. They have also been accused of not setting up the proper systems to keep kids safe online. In newly unsealed court filings in a big child-safety lawsuit, Meta’s former head of safety and well-being, Vaishnavi Jayakumar, testified that Meta had a “17-strike policy” for accounts engaged in sex trafficking. I think most parents agree this is abhorrent. Meta has denied this characterization, saying it removes accounts suspected of human trafficking immediately and that the plaintiffs’ brief presents a misleading picture but Meta has proven itself very hard to trust.
I’ve also been outspoken against OpenAI because they are not doing enough to protect our kids. They are making all of the same mistakes that social media made. They don’t think ahead and prioritize safety, and they continue to launch products that are harmful to kids. Their parental controls are reactive on the heels of lawsuits (launched after a family alleging OpenAI “actively helped” their son explore suicide methods), are the bare minimum, and are insanely easy to circumvent – just open up a new tab as an anonymous user and don’t use the account linked to your parents account. That’s not good enough! AI responses should not be given on any questions that are inappropriate for young kids without proper age verification and parental controls in place.
There are big opportunities for companies to use AI to build safer online experiences for kids and win over the trust of parents. When thoughtfully designed and implemented, AI can serve as a powerful tool – one that scales to the size of today’s digital ecosystems – supporting kids as they explore, grow, and connect safely online.
