Zombie stats: Where AI gets its numbers
claude gemini google perplexity
| Source: HN | Original article
Around 40% of stats quoted by AI assistants come directly from a single study, while an additional 8% are derived from studies with altered numbers or context.
A new study from Trakker Research has mapped the provenance of the statistics that large‑language‑model assistants quote. By analysing 1,285 figures supplied by ChatGPT, Perplexity, Gemini, Claude and Google AI Overviews, the researchers found that only 40 % of the numbers could be traced back to a study that actually reported the same figure. An additional 8 % reached a source but with the number altered or taken out of context. The remaining citations fell into dead ends: roughly 40 % led to locked, removed or otherwise silent pages, while 11 % could not be located at all. The study labels these unverified figures “zombie statistics” – numbers that persist in AI output despite lacking a verifiable origin.
The findings matter because AI assistants are increasingly used for quick fact‑checking and decision‑making. When a model confidently repeats a statistic that cannot be substantiated, users may accept false or distorted information as truth, undermining trust in AI‑driven research tools and amplifying misinformation. The prevalence of zombie stats also highlights a gap in current model training and retrieval pipelines, which often prioritize fluency over source verification.
Going forward, observers will watch how AI providers respond. Potential developments include tighter integration of citation‑checking mechanisms, stricter sourcing standards for retrieval‑augmented generation, and third‑party tools that flag unverifiable numbers. Regulators and industry bodies may also consider guidelines for transparent sourcing in conversational AI. The Trakker report underscores the urgency of building safeguards before the habit of citing “statistics” without proof becomes entrenched in everyday AI interactions.
Sources
Back to AIPULSEN