Reviso: Visual Evidence Varies in Importance for Video Captioning
multimodal
| Source: HF Papers | Original article
Researchers from Peking University, KlingAI Research, HKUST and other institutions have unveiled **Reviso**, a new agentic video‑captioning framework that deliberately chooses which visual evidence to attend to when generating descriptions. The system, detailed in a paper released on November 3, tackles a known shortcoming of multimodal large language models (MLLMs): they often allocate equal weight to every visual detail, causing captions to linger on trivial elements while missing the core events and context of a clip.
Reviso addresses this by employing hierarchical planning and active perception. Instead of processing an entire video at full resolution, the model first sketches a high‑level plan, then concentrates its limited 128‑frame inspection budget on the segments deemed most informative for the caption. Experiments on four established video‑captioning benchmarks show that Reviso outperforms open‑source baselines, delivering captions that better capture key moments and that prove more useful for downstream video‑understanding tasks.
The development matters because accurate video captions are essential for accessibility, content moderation, search and recommendation systems. By teaching models to “look where it counts,” Reviso not only improves narrative fidelity but also reduces unnecessary computation, a critical factor as video data scales.
Going forward, the community will watch for broader validation of Reviso’s approach on larger, more diverse video corpora and its integration into commercial multimodal pipelines. Further research may explore extending the hierarchical planning mechanism to other vision‑language tasks, such as video‑question answering or real‑time summarisation, potentially setting a new standard for selective visual attention in AI systems.
Sources
Back to AIPULSEN