New Benchmark Tests How Underwater Image Quality Metrics Align with Human Perception
benchmarks
| Source: HF Papers | Original article
Researchers benchmark 73 underwater image quality metrics against human choices, simple edits and generative drift to test alignment with human perception.
Researchers have released a new benchmark that pits 73 handcrafted and learned no‑reference image quality assessment (NR‑IQA) metrics for underwater photography against human judgments. Because clean reference images are rarely available beneath the surface, developers of underwater enhancement algorithms have long depended on NR‑IQA scores to gauge progress. The study asks a simple but crucial question: do these automated scores actually reflect what people perceive as higher‑quality images?
To answer it, the team assembled a dataset that combines three evaluation strands. First, they collected human choices on a range of enhanced underwater photos, establishing a ground‑truth perception baseline. Second, they introduced “simple edits” – controlled adjustments such as contrast or color balance – to isolate how metrics respond to straightforward improvements. Third, they incorporated “generative drift,” where state‑of‑the‑art generative models produce more dramatic, sometimes unrealistic, enhancements. By running all 73 metrics across these conditions, the authors measured correlation with the human rankings.
The findings reveal a mixed picture. Some traditional handcrafted metrics align closely with human preferences on modest edits, while many learned metrics struggle, especially when generative drift introduces artifacts that humans readily spot. This discrepancy highlights a gap in current evaluation practices and suggests that relying solely on NR‑IQA scores may mislead developers about real‑world performance.
Going forward, the benchmark offers a testbed for refining existing metrics and designing new ones that better capture human perception under water. Researchers and industry players are likely to use the dataset to validate future enhancement models, and we may see a shift toward hybrid evaluation frameworks that blend algorithmic scores with crowd‑sourced judgments. Monitoring upcoming conferences and journal releases for metric improvements will be essential to track progress in this niche yet growing field.
Sources
Back to AIPULSEN