YANAMI Unveils Large‑Scale Multimodal Dataset and Benchmark for Multi‑Turn Stateful Interaction
benchmarks multimodal
| Source: HF Papers | Original article
Researchers introduce YANAMI, a large‑scale multimodal dataset and benchmark designed for multiturn, stateful interaction, addressing the single‑turn bias of existing unified multimodal datasets.
A team of researchers from HKUST, Shanghai AI Lab and several partner institutions has released YANAMI, the first large‑scale multimodal dialogue dataset and benchmark designed for stateful, multi‑turn interaction. The collection, dubbed YANAMI‑400K, comprises 400 000 dialogues that span more than 2 million images and 2.6 million conversational turns. A companion benchmark, YANAMI‑Bench, evaluates models on 400 multi‑turn dialogues, jointly testing Dialogue State Understanding (the ability to track evolving intents, references and visual context) and Visual Realisation (generation, editing or manipulation of images in line with the dialogue).
The release addresses a clear gap in current Unified Multimodal Models (UMMs). Existing multimodal datasets and benchmarks are dominated by single‑turn tasks, and even the newer multi‑turn collections often allow each turn to be solved in isolation. YANAMI forces models to maintain and act on a persistent dialogue state, a capability essential for truly interactive AI assistants, robotics controllers and mixed‑reality agents.
Initial experiments reported in the paper show that fine‑tuning the BAGEL model on YANAMI‑400K yields measurable gains: image‑editing performance rises by 10.3 %, generation by 2.0 % and overall stateful interaction by 13.23 %, without sacrificing general visual understanding. The results underline that current UMMs still lag on this more demanding setting, highlighting a fertile research frontier.
Going forward, the community will watch for broader adoption of YANAMI as a standard testbed. Upcoming leaderboards are likely to feature other leading multimodal architectures, and extensions of the benchmark could incorporate additional domains such as code generation or cybersecurity, echoing recent efforts like SemiAnalysis’s GDPx benchmark. Success on YANAMI could become a key indicator of an AI system’s readiness for real‑world, context‑aware multimodal interaction.
Sources
Back to AIPULSEN