Study Traces Psychometric Inference in Large Language Models
inference llama
| Source: ArXiv | Original article
Researchers examine whether large language models internally encode psychometric constructs, testing 14 variants from two families using a cross‑persona approach.
A new arXiv pre‑print, arXiv:2611.12937v1, reports the first systematic attempt to map psychometric inference inside large language models (LLMs). The authors examined 14 variants from two model families – Llama 3 and Qwen 2.5 – spanning sizes from 0.5 billion to 14 billion parameters and covering both base and instruction‑tuned versions. Using a cross‑persona experimental paradigm, they traced the relationship between behavioural outputs (e.g., questionnaire‑style responses) and the models’ internal activation patterns.
The study finds that a structured “psychometric geometry” emerges in activation space as models grow larger and receive instruction tuning. This geometry predicts how strongly behavioural signals are amplified: within the instruction‑tuned group the correlation reaches r = 0.90, and even after controlling for model size the partial correlation remains substantial (r = 0.69). The authors also show that targeted steering of the internal representation can modulate the observed psychometric behaviour, suggesting that the latent structure is not merely an artifact of output generation.
Why it matters is twofold. First, it moves the debate beyond whether LLMs can mimic psychometric patterns to whether they actually encode psychological constructs internally, a question central to interpretability and alignment. Second, the findings open the door to using LLMs as scalable tools for psychological measurement, echoing earlier work such as the Nature‑published psychometric framework for evaluating personality‑like traits (Dec 2025).
The next steps will likely involve extending the analysis to larger, commercial models, testing the robustness of the internal geometry across tasks, and exploring practical applications in personality assessment or bias mitigation. Watch for follow‑up studies that combine this internal mapping with real‑world deployment scenarios, as well as any attempts to harness the identified geometry for more controllable AI behaviour.
Sources
Back to AIPULSEN