OpenAI unveils new image classifier and ChatGPT warnings to combat CSAM
openai
| Source: TechCrunch | Original article
OpenAI has launched a new image classifier and introduced a 24‑hour account ban as a warning for users attempting to access CSAM content via ChatGPT.
OpenAI unveiled a new image‑classification system designed to sift through the flood of deep‑fake child sexual abuse material that is overwhelming tip‑lines for investigators. At a press briefing in Washington, DC, the company said the tool can more reliably flag abusive imagery of children—including synthetic or manipulated content—allowing law‑enforcement and child‑protection teams to focus on the most actionable files. The announcement positions the classifier as a “noise‑reduction” aid for investigators who must wade through massive volumes of potentially harmful media.
Alongside the classifier, OpenAI rolled out a user‑facing safeguard that displays a warning screen whenever a prompt or query appears to be heading toward material that sexualises children. If a user proceeds, the system automatically suspends the account for 24 hours, echoing the short‑term ban introduced earlier this year. The dual approach couples proactive detection with an explicit deterrent, aiming to curb both accidental exposure and deliberate attempts to retrieve illegal content.
The move marks OpenAI’s most visible step yet toward embedding content‑moderation technology into its consumer‑facing products and aligns the firm with tightening global regulations that increasingly hold AI providers accountable for misuse. By targeting deep‑fakes—a growing concern as generative models make synthetic abuse easier to produce—OpenAI hopes to stay ahead of a threat that could otherwise swamp child‑abuse hotlines.
Observers will watch how accurately the classifier distinguishes genuine abuse from false positives, especially given the technical challenges of spotting AI‑generated imagery. Regulators are likely to scrutinise whether the warning screens and suspension policy meet emerging legal standards for AI‑driven content control. If successful, the tools could set a baseline for other AI companies seeking to balance open access with robust safeguards against child sexual abuse material.
Sources
Back to AIPULSEN