🔗 Share this article

Technology companies and child protection organizations will receive permission to assess whether AI tools can produce child exploitation material under new UK legislation.

Substantial Increase in AI-Generated Illegal Content

The announcement came as findings from a safety monitoring body showing that reports of AI-generated CSAM have increased dramatically in the last twelve months, rising from 199 in 2024 to 426 in 2025.

New Legal Framework

Under the changes, the authorities will permit designated AI developers and child safety organizations to inspect AI systems – the underlying systems for conversational AI and image generators – and ensure they have sufficient safeguards to stop them from creating images of child sexual abuse.

"Ultimately about stopping abuse before it occurs," declared Kanishka Narayan, noting: "Experts, under rigorous protocols, can now identify the risk in AI models promptly."

Tackling Regulatory Obstacles

The amendments have been introduced because it is against the law to create and possess CSAM, meaning that AI developers and other parties cannot create such images as part of a evaluation regime. Until now, officials had to wait until AI-generated CSAM was published online before dealing with it.

This legislation is designed to preventing that issue by helping to halt the production of those materials at source.

Legislative Framework

The changes are being introduced by the government as modifications to the criminal justice legislation, which is also implementing a ban on owning, producing or distributing AI models developed to create child sexual abuse material.

Practical Impact

This recently, the minister toured the London headquarters of Childline and heard a mock-up call to advisors involving a report of AI-based abuse. The interaction depicted a adolescent seeking help after facing extortion using a explicit AI-generated image of themselves, created using AI.

"When I hear about children facing extortion online, it is a source of intense anger in me and justified concern amongst parents," he said.

Concerning Statistics

A leading internet monitoring organization stated that cases of AI-generated abuse content – such as webpages that may contain multiple files – had significantly increased so far this year.

Instances of category A content – the gravest form of exploitation – rose from 2,621 images or videos to 3,086.

• Girls were overwhelmingly victimized, making up 94% of illegal AI images in 2025
• Depictions of infants to toddlers increased from five in 2024 to 92 in 2025

Sector Reaction

The law change could "constitute a crucial step to guarantee AI products are safe before they are launched," commented the chief executive of the online safety foundation.

"Artificial intelligence systems have enabled so survivors can be targeted repeatedly with just a simple actions, providing offenders the ability to make potentially limitless amounts of advanced, photorealistic child sexual abuse material," she added. "Content which further commodifies survivors' trauma, and renders children, especially female children, more vulnerable both online and offline."

Support Interaction Data

The children's helpline also published details of support sessions where AI has been referenced. AI-related risks discussed in the conversations include:

• Employing AI to evaluate weight, body and looks
• Chatbots dissuading young people from talking to trusted guardians about harm
• Facing harassment online with AI-generated material
• Online extortion using AI-manipulated images

Between April and September this year, the helpline conducted 367 counselling interactions where AI, conversational AI and associated terms were mentioned, four times as many as in the equivalent timeframe last year.

Half of the mentions of AI in the 2025 interactions were connected with mental health and wellness, including using chatbots for assistance and AI therapy applications.