Exclusive: US military narrowly avoided disaster after using AI for fabricated intelligence, sources say
| Source: Mastodon | Original article
US military nearly acted on AI‑generated false intel claiming a Chinese vessel was carrying nuclear weapons components, prompting a rapid interception plan.
A U.S. Special Operations Command unit circulated an intelligence brief this spring that claimed a Chinese vessel in the Middle East was carrying components for a nuclear‑weapons programme. The assessment, generated by an artificial‑intelligence tool, triggered immediate plans to board the ship and intercept the cargo. Within hours, senior officials flagged the report as “entirely false,” halting the operation before any force was deployed. One source said the mistake “almost started a war,” warning that an attack on a Chinese ship could have spiralled into a broader armed conflict amid the ongoing war with Iran.
The incident underscores the growing danger of relying on AI‑driven analysis for high‑stakes military decisions. Unlike traditional human‑crafted intelligence, the AI system produced a hallucinated claim that appeared credible enough to set operational plans in motion. The episode arrives on the heels of a similar misstep we reported on 19 September, when a false AI‑generated report about Chinese nuclear components nearly prompted a U.S. strike. Together, the cases illustrate how relatively immature generative‑AI technology can amplify the risk of miscalculation in volatile geopolitical environments.
Policymakers and defence leaders are now likely to scrutinise the chain of command that allowed the AI output to reach operational planners. Watch for internal investigations, possible revisions to AI‑use protocols, and congressional hearings on oversight of machine‑generated intelligence. Tech firms that supply the underlying models may also face pressure to improve transparency and reliability, especially as the military seeks to balance the speed of AI assistance with the need for rigorous verification. The close call could become a catalyst for tighter controls on AI deployment in national‑security contexts.
Sources
Back to AIPULSEN