WorldBench
A 2,000-question multimodal reasoning benchmark built to maximize visual diversity, not just task diversity, across seven real-world visual domains.
What It Tests
WorldBench is a multimodal reasoning benchmark from researchers at Princeton University (Yida Yin, Harish Krishnakumar, Chung Peng Lee, Boya Zeng, Wenhao Chai, Shengbang Tong, Wenhu Chen, Hu Xu, Xingyu Fu, Gabriel Sarch, Aleksandra Korolova, Zhuang Liu) designed to test whether multimodal large language models (MLLMs) can reason reliably across the full breadth of visual settings they encounter in the real world, rather than just a wide range of task formats. The authors argue that prior multimodal benchmarks expand task types (VQA, captioning, chart reading, etc.) while sampling images narrowly from a small number of iconic, object-centric sources, leaving a gap in visual diversity that lets models look strong without actually generalizing.
To close that gap, the authors built a large taxonomy of roughly 2,000 visual concepts spanning seven domains (Living Things, Objects, Scenes, Digital World, Academics, Documents/Charts/Tables, and Agents), then curated high-quality, deliberately non-iconic images for each concept -- images that represent a concept in a realistic, cluttered, or unusual context rather than a clean textbook photo. Questions were then hand-written through a structured trial-and-error process specifically to expose cases where frontier MLLMs answer incorrectly despite the task being natural and intuitive for a human. The result is 2,000 curated multiple-choice questions, each paired with an image.
What differentiates WorldBench is its explicit focus on measuring and maximizing visual diversity itself: the authors validate this with embedding-based diversity metrics (effective rank and participation ratio across three vision encoders) and human pairwise (Bradley-Terry) diversity ratings, both showing WorldBench is more visually diverse than existing 'diverse' benchmarks. On the evaluation side, the headline trajectory is that the benchmark is far from saturated -- the strongest model evaluated (Gemini-3.1-Pro) reaches only 64.0% average accuracy, and no model exceeds 75% accuracy in any single domain, with several open-source models performing only marginally above chance on a 4-option multiple-choice task.
Task Anatomy
How a single task is structured.
Example Tasks
2 real examples from the benchmark.
Living Things domain (representative format, not a verbatim quoted item)
Problem / Input
A non-iconic photograph depicting a living-thing concept from the taxonomy in a realistic, uncontrolled setting (e.g., partially occluded, camouflaged, or shown in an atypical context) rather than a clean, centered reference photo, accompanied by a 4-option multiple-choice question probing a specific visually-grounded detail (e.g., identity, attribute, count, or relationship) that requires actually looking at the image rather than guessing from common priors.Constructed from the benchmark's documented construction methodology and dataset schema (image, question, option_a-d, answer, category, domain fields); the paper/project page/GitHub/HuggingFace pages accessed did not expose the literal text of individual questions, so no specific question wording is quoted here.
Documents/Charts/Tables (DCT) domain (representative format, not a verbatim quoted item)
Problem / Input
Same caveat as above: format is confirmed via the HuggingFace dataset card schema and GitHub README, but exact question text was not available in the sources reviewed.
Leaderboard Results
Model scores sorted by performance.
15 results
| # | Model | Score |
|---|---|---|
| 1 | Gemini 3.1 Pro | 64.0% avg accuracy |
| 2 | Gemini 3 Flash | 61.8% avg accuracy |
| 3 | Qwen3.5-VL-Plus (Thinking) | 59.3% avg accuracy |
| 4 | GPT-5.4 Thinking | 58.2% avg accuracy (Thinking, high effort) |
| 5 | Qwen3.5-VL-27B | 56.6% avg accuracy |
| 6 | Claude Opus 4.7 | 53.7% avg accuracy |
| 7 | Grok 4.2 | 53.3% avg accuracy |
| 8 | GPT-5.4 Thinking | 53.0% avg accuracy (Thinking, low effort) |
| 9 | Qwen3.5-VL-35B-A3B | 52.9% avg accuracy |
| 10 | Kimi K2.5 | 52.5% avg accuracy |
| 11 | Gemma 4 31B | 49.7% avg accuracy |
| 12 | Qwen3.5-VL-Plus (Instruct) | 48.7% avg accuracy |
| 13 | GLM-4.6V | 42.5% avg accuracy |
| 14 | InternVL3.5 | 41.2% avg accuracy |
| 15 | Gemma 4 E4B | 34.6% avg accuracy |
V= Self-reported by the model's creator, not independently verified
Score Over Time
Performance progression across model generations.
Key Findings
The best-performing model overall, Gemini-3.1-Pro, reaches only 64.0% average accuracy across the 2,000 questions, and no model exceeds 75% accuracy in any single one of the 7 domains.
The best open-source/open-weight model, Qwen3.5-VL-27B, reaches 56.6% average accuracy, trailing the top proprietary model by about 7.4 points.
Several smaller open models (e.g., Gemma-4-E4B at 34.6%, InternVL-3.5 at 41.2%) perform only marginally above chance level on the 4-option multiple-choice format, revealing large gaps in robustness to visually diverse, non-iconic imagery.
Embedding-based diversity metrics (effective rank / participation ratio across three vision encoders) and human pairwise Bradley-Terry ratings from 12 raters both confirm WorldBench achieves higher visual diversity than prior 'diverse' multimodal benchmarks.
The authors explicitly frame the benchmark's headline result as 'WorldBench Remains Far From Saturated,' positioning visual (not just task) diversity as the harder, still-unsolved axis of multimodal evaluation.
Controversies & Caveats
Known limitations and criticisms.
WorldBench was only posted to arXiv in June 2026, roughly a month before this research was conducted; no independent third-party audits, contamination analyses, or critical replications were found in the sources reviewed, so current scores rest entirely on the authors' own evaluation harness.
Some leaderboard entries (e.g., GPT-5.4-Thinking at 'high' vs 'low' reasoning effort) show non-trivial score swings from inference-time settings alone, which could complicate cross-model comparisons if effort settings aren't standardized in future reporting.