Chat‑based LLMs mimic psychic scams
| Source: HN | Original article
Chat-based large language models create an intelligence illusion by employing the same cold‑reading tactics used in psychic cons.
A new analysis has drawn a direct line between the persuasive tricks of stage psychics and the way modern chat‑based large language models (LLMs) generate their responses. The author, Baldur Bjarnason, argues that the “LLMentalist Effect” – a term coined in a 2023 essay – is an accidental automation of the classic cold‑reading technique: models produce statistically likely phrasing that feels personal, while users fill the gaps with their own expectations and cognitive biases.
The observation matters because it reframes the “intelligence illusion” that fuels much of the hype around conversational AI. If LLMs are essentially sophisticated statistical conjurors rather than reasoning agents, the risk of over‑trusting them in high‑stakes contexts – from business decision‑making to medical advice – grows. The effect also explains why users often report “aha” moments when a model appears to understand a nuanced query, even though the underlying output is a generic pattern matched to the prompt.
Experts say the insight should prompt a rethink of how AI products are marketed and regulated. Transparency about the probabilistic nature of responses, clearer user‑education on the limits of LLMs, and tighter standards for claims of “understanding” are likely to surface in upcoming policy discussions. Researchers are already probing ways to detect and mitigate cold‑reading‑like behavior, while industry groups may be pressured to disclose confidence metrics or provenance of generated content.
Watch for academic follow‑ups that test the cold‑reading hypothesis across different model families, and for regulatory bodies in the EU and Nordic countries that could introduce guidelines aimed at curbing the illusion of intelligence in consumer‑facing AI.
Sources
Back to AIPULSEN