AI risk: Tackle child sexual abuse imagery crisis

AI-generated child sexual abuse material (CSAM) is rapidly becoming one of the most dangerous threats to children online.

With the rise of generative artificial intelligence, criminals are now able to create hyper-realistic abuse images and videos of children who do not exist or digitally manipulate existing materials to appear newly abusive.

The result is a troubling loophole in the global child protection system—one that current laws and safeguards are not prepared to handle.

What makes this crisis particularly urgent is that no real child needs to be present for harm to be done. The AI images often draw from databases of existing child images, sometimes of past abuse survivors, and render new explicit content. Survivors have spoken out about the trauma of knowing their images are being manipulated and repurposed without their consent.

The psychological toll is real and the legal consequences remain unclear in many jurisdictions.

Europol’s recent global operation that led to the arrest of 25 individuals involved in creating and sharing AI-generated CSAM is evidence of how widespread and organised this abuse has become.

Investigators found that offenders were using publicly available AI tools to create CSAM that could evade conventional detection systems. Unlike traditional CSAM, which can be flagged through hashing technologies, AI-generated abuse content is novel with each iteration—making it nearly impossible to trace with current tools.

Some of the content mimics real children so convincingly that even trained analysts cannot distinguish between fabricated and authentic abuse.

Predators use these tools to create and distribute material that is difficult to regulate, especially on the dark web and in encrypted communities where much of this activity thrives.

In the UK, the National Crime Agency has labelled this development a “nightmare scenario” and is pushing for urgent reforms. Current laws are typically structured around the protection of real children depicted in abuse imagery, but AI muddies those definitions.

Read: Kenya’s AI dilemma: Experts call for incentives as State signals censorship

If a child doesn’t exist, is the material still illegal? Increasingly, experts argue yes—because the intent, effect, and impact are equally harmful. Some jurisdictions have begun amending their laws to criminalise AI-generated CSAM, but enforcement remains difficult.

The most dangerous implication of this technology is scale. One offender can now use AI to generate thousands of images in minutes.

That content is then circulated globally, further overwhelming already strained detection systems.

As technology becomes more sophisticated and accessible, the problem will grow exponentially as detection lags behind creation.

In countries like Kenya, where Internet access is rising rapidly, awareness of AI-generated threats is low. A recent report revealed a growing gap in online child protection.

While digital literacy campaigns have helped raise general awareness, many parents, teachers, and policymakers remain unaware of the threats posed by synthetic content.

AI-generated CSAM adds complexity to an already under-resourced system and calls for an urgent review of Kenya’s digital child safety policies.

Legal reform is only one part of the solution. Technology companies must also step up by developing tools that can detect AI-generated content before it spreads. This includes deploying machine learning models to identify patterns consistent with synthetically created images.

Startups and research labs are beginning to test such systems, but they need support, regulation, and integration with national and global efforts.

International collaboration is essential because AI-generated CSAM crosses borders in seconds, a fragmented approach will fail.

Countries must coordinate on shared databases, detection technologies, legal definitions, and law enforcement protocols.

Just as financial institutions have developed shared fraud alert systems, tech platforms and governments must co-create real-time alert systems for abusive AI content.

AI can also be part of the solution. The same tools that are used to generate harmful content can be adapted to detect it. By analyzing patterns, metadata, and inconsistencies in rendering, AI can flag suspect content with high accuracy.

This technology, combined with human oversight, offers a path forward—if the political and corporate will is there to invest in it.

We must also include survivors in policy design. Their experiences provide essential insight into how technology perpetuates harm. Some have advocated for stronger protections against the manipulation of personal images, especially those taken during childhood.

Laws must reflect these voices by banning not just the distribution of synthetic CSAM, but also its creation, possession, and even prompting through AI models.

The battle against AI-generated CSAM is only beginning. If we act now—with urgency, coordination, and innovation—we can prevent this crisis from becoming irreversible. But if we delay, the cost will be measured in more than just terabytes of illicit data. It will be counted in lives disrupted, trauma reignited, and justice denied.

AI must be governed by a clear moral boundary. Children—real or synthetically depicted—must be off-limits. Technology must never become a weapon of exploitation. Governments, companies, and citizens all have a role to play in ensuring that AI is used to protect the vulnerable—not empower the abusive.

The writer is a Machine Learning Researcher, Technology Policy Analyst and a Columnist