Burak Satar

Burak Satar

Research Scientist @ SMU | PhD @ NTU

Cultural VLM Resources

A curated list of benchmarks and datasets for measuring how well vision-language models (and multimodal AI generally) handle cultural and geographic diversity. I maintain this page as the field moves; it accompanies my post Why Vision-Language Models Fail Outside the West.

Missing something? Email me and I will add it.

Last updated: July 2026.

Southeast Asia

  • Seeing Culture (EMNLP 2025): 1,065 images of 138 cultural artifacts from seven SEA countries, 3,000+ questions. Two-stage evaluation: answer a culturally grounded visual question, then segment the artifact as visual evidence of the reasoning. Dataset on HuggingFace, project page. Disclosure: this is our work.
  • SEA-VL (ACL 2025): a 1.28M-image culturally relevant vision-language dataset for Southeast Asia, with a systematic comparison of crowdsourcing, crawling and synthetic generation as collection methods.
  • SEACrowd (EMNLP 2024): a data hub and benchmark suite consolidating standardized corpora (text, image, audio) across roughly 1,000 Southeast Asian languages, with evaluations on 36 indigenous languages.

Cultural visual reasoning and VQA

  • MaRVL (EMNLP 2021): visually grounded reasoning built around concepts and images selected by native speakers of Indonesian, Mandarin, Swahili, Tamil and Turkish.
  • GD-VCR (EMNLP 2021): geo-diverse visual commonsense reasoning; models score notably lower on East Asian, South Asian and African images, with the biggest gaps on culture-specific scenarios.
  • CVQA (NeurIPS 2024 D&B): culturally grounded VQA from 30 countries in 31 languages and 13 scripts.
  • CulturalVQA (EMNLP 2024): probes frontier VLMs on cultural understanding across regions; strong on North American content, substantially weaker on African contexts.

Food and everyday culture

  • FoodieQA (EMNLP 2024): fine-grained multimodal QA on regional Chinese food culture; open-source VLMs trail humans by roughly 41 percent on multi-image questions.
  • WorldCuisines (NAACL 2025, Best Theme Paper): 1M+ VQA data points on global cuisines across 30 languages, including adversarial location contexts.
  • BLEnD (NeurIPS 2024 D&B): everyday cultural knowledge across 16 regions and 13 languages. Text-only (LLM benchmark), useful for separating the knowledge gap from the perception gap.

Geographic robustness of visual recognition

  • Does Object Recognition Work for Everyone? (CVPR Workshops 2019): commercial recognition systems are markedly less accurate on household items from low-income, non-Western households (Dollar Street).
  • GeoDE (NeurIPS 2023 D&B): 61,940 images of 40 object classes crowdsourced directly from six world regions, built to evaluate geographic disparities without web-scraping bias.

Foundational reading

  • The weirdest people in the world? (Henrich, Heine and Norenzayan, Behavioral and Brain Sciences 2010): the WEIRD sampling critique that the AI version of this problem echoes.