We Still Don’t Know How People Really Use AI
anthropic claude openai
| Source: MIT Tech Review | Original article
AI firms such as Anthropic and OpenAI release selective usage reports for Claude and ChatGPT, leaving researchers without independent verification of how users actually employ the tools.
AI firms are publishing usage statistics, but researchers say the picture remains incomplete. Anthropic and OpenAI regularly release reports on how users interact with their models—Claude and ChatGPT respectively—yet the data they share is selective, according to a growing chorus of scholars. “There is no independent source to corroborate it,” notes Anka Reuel, a Computer Science PhD candidate, highlighting a gap in external verification.
The lack of independent data matters because usage metrics shape everything from product roadmaps to public policy. Companies tout high engagement numbers to attract investors and justify scaling, while regulators rely on those figures to assess societal impact, safety risks, and potential bias. Without an unbiased benchmark, it is difficult to gauge whether AI tools are being employed responsibly, whether certain demographics are over‑ or under‑represented, or how emerging use‑cases—such as automated content creation or decision‑support systems—might be influencing broader economic and cultural trends.
The issue also raises questions about accountability. If firms can cherry‑pick favorable metrics, stakeholders—including users, policymakers, and competitors—are left without a clear view of real‑world effects. This opacity could hinder efforts to develop standards for ethical AI deployment and to monitor unintended consequences.
Looking ahead, the AI community is watching for moves toward third‑party audits and standardized reporting frameworks. Legislative bodies in Europe and North America have signaled interest in mandating more transparent disclosures, while academic groups are exploring independent measurement tools. The next wave of scrutiny will likely focus on whether industry players will open their data pipelines to external reviewers or continue to rely on self‑reported figures, a decision that could shape trust in AI across the Nordics and beyond.
Sources
Back to AIPULSEN