Burak Satar

Burak Satar

Making vision-language models culturally aware at SMU, PhD graduate from NTU.

Cultural VLM Resources

A curated list of 89 benchmarks, datasets and studies for measuring how well vision-language models (and multimodal AI generally) handle cultural and geographic diversity: image, video, text-to-image and text-to-video. I maintain this page as the field moves; it accompanies my post Why Vision-Language Models Fail Outside the West.

Every entry was checked against its paper, the venue proceedings and its dataset page; the last full check was on 17 September 2026. Entries are newest first within each section. Data notes: Data gated means a Hugging Face access agreement, Data on request means a form or email to the authors, Data coming soon means the authors have announced a release, and No public data found means no release could be located.

The same list is on GitHub as awesome-cultural-vlm, which is where to send additions and corrections. You can also email me.

Last updated: September 2026.

Southeast Asia

Everything on this list that is built from Southeast Asian images, videos or prompts, whatever the task. Neighboring traditions here share surface features (rice, water, gold, drums), so fine-grained discrimination is the test.

  • Cultural Moment Benchmark (EMNLP 2026 Main, oral): Three-stage video benchmark of 306 expert-curated cultural concepts from seven Southeast Asian countries: naming, recognizing among video moments, and temporal localization. Public sample; hidden test set held back. Disclosure: this is our work. Dataset, Code, Project page.
  • GG-EZ (arXiv, 2026): Regional data filtering plus model merging to adapt LVLMs, SDXL and SigLIP-2 to Southeast Asia, gaining 5 to 15 percent in cultural relevance. Dataset.
  • Rice-VL (arXiv, 2025): ASEAN cultural VQA benchmark: over 28,000 human-curated questions on 7,000 images and 1,000 grounding boxes across 11 countries and 14 categories. No public data found.
  • SEA-VL (ACL 2025 Main): Compares crowdsourcing, crawling and image generation for collecting culturally relevant data, yielding 1.28M Southeast Asian images across 11 countries. Dataset, Code.
  • Seeing Culture (EMNLP 2025 Main): Two-stage benchmark: multiple-choice VQA with image options then segmentation of the cultural artifact, 1,065 images, 138 artifacts, 3,178 questions, seven Southeast Asian countries. Disclosure: this is our work. Dataset, Code, Project page.
  • VietMEAgent (FAIR 2025 conference): Vietnamese cultural VQA method pairing cultural object detection, program generation and a knowledge base for dual-modality explanations, with a 28,484-sample dataset over 12 categories. Dataset.
  • SEA-VQA (ACL 2024 Workshop (ALVR)): Culture-centric VQA on UNESCO cultural heritage images from 8 Southeast Asian countries; GPT-4 and Gemini score substantially lower than on A-OKVQA. Dataset.
  • SEACrowd (EMNLP 2024 Main): Data hub of 498 datasheets and 399 dataloaders for Southeast Asian languages across text, image and audio, with a benchmark on 13 tasks. Dataset, Code, Project page.

Multi-country VQA and cultural knowledge

Benchmarks that span many countries or regions. Use these to compare models across cultures at once.

  • CulturalMenuBench (EMNLP 2026 Findings): 4,870 culinary items in 10 languages and 18 regions pair dish and cooking-step images with text to test regional cuisine attribution beyond recognition. Data coming soon. Code.
  • BLEnD-Vis (EACL 2026 Main): Extends BLEnD to VQA with 4,916 generated images and over 21,000 multiple-choice questions across 16 regions, testing rephrasing and cross-modal robustness. Dataset, Code.
  • C3B (ICLR 2026): Comic-image benchmark of 2,220 images and 18,789 QA pairs with recognition, cultural-conflict and multilingual generation tasks; 11 open-source MLLMs evaluated. Dataset, Project page.
  • ConfusedTourist (CVPR 2026 Findings): Adversarial suite of 5,451 images of cuisine, attire and musical instruments from 57 countries with stacked or generated conflicting cues; accuracy drops heavily. Dataset, Code.
  • CultureMix (CVPR 2026): 23k generated food VQA images mixing dishes and backgrounds from 30 countries, testing whether VLMs keep each element’s cultural identity. Dataset.
  • GIMMICK (ACL 2025 Findings): Six tasks on 728 cultural events and facets from 144 countries in six regions, with 6,887 images and 993 videos; 31 models evaluated. Data gated. Dataset, Code.
  • MMAC (ACL 2026 Main): Human-curated multiple-choice benchmark of 27,000 questions aligned across text, image and speech, covering 8 Asian countries and 10 languages, with cross-modal consistency tests. Dataset.
  • ALM-bench (CVPR 2025): 22,763 questions in 100 languages from 73 countries across 13 cultural and 6 generic categories, in four question formats. Dataset, Code, Project page.
  • CROPE (NAACL 2025 Main): VQA benchmark of 1,060 binary questions on culture-specific versus common concepts, testing whether VLMs adapt from in-context text and image descriptions. Dataset, Code.
  • CulturalVQA (EMNLP 2024 Main): 2,378 questions on 2,328 images from 11 countries across five continents on clothing, food, drinks, rituals and traditions, answered by annotators familiar with each culture. Dataset, Project page.
  • CVQA (NeurIPS 2024 Datasets and Benchmarks): Multilingual VQA benchmark of 10,374 four-way questions on culturally driven images from 30 countries in 31 languages, built with native speakers and cultural experts. Dataset, Project page.
  • WorldCuisines (NAACL 2025 Main): Multilingual food VQA benchmark with over 1 million instances in 30 languages, asking dish names and origins for 2,414 dishes from 189 countries. Dataset, Code, Project page.

Country and region VQA and cultural knowledge

Benchmarks built for one country or region, usually by people from it. Grouped by region.

South Asia

  • BanglaProtha (WACV 2026): VQA dataset of Bengali cultural images with native Bengali questions and semantically similar distractors, evaluating VLMs across prompting, fine-tuning and cultural aspects. Dataset, Code.
  • BanglaVerse (EMNLP 2026 Findings): 1,152 Bengali-culture images for VQA and captioning, expanded into four languages and five Bangla dialects (about 32.2K items) across nine domains. Dataset, Code, Project page.
  • TAB-VLM (ACL 2026 Findings): 600 questions over 1,600 Indian artifacts from prehistoric to modern periods testing temporal reasoning; best of ten models (GPT-5.2) reaches 58.7%. Dataset, Code, Project page.
  • DRISHTIKON (EMNLP 2025 Main): Indian-culture VQA benchmark of 64,288 instances in 15 languages covering all states and union territories across 16 cultural themes. Dataset, Code.
  • IndicVisionBench (ICLR 2026): India-centric benchmark: about 5K images and 37K+ QA pairs for VQA, OCR and multimodal translation in English and 10 Indian languages, 13 cultural topics. Dataset, Code.

East Asia

  • Hanfu-Bench (EMNLP 2025 Main): 1,192 images of 496 Hanfu outfits across Chinese dynasties for cultural VQA and image transcreation; closed VLMs trail human experts by 10%. Data gated. Dataset, Code.
  • TaiwanVQA (NeurIPS 2025 Datasets and Benchmarks): Taiwan-specific VQA benchmark of 2,736 images and 5,472 questions on food, signs, festivals and landmarks, plus augmentation to improve cultural reasoning. Dataset, Code, Project page.
  • CVLUE (AAAI 2025): Chinese vision-language benchmark with native-speaker-selected images over 92 categories, covering image-text retrieval, VQA, visual grounding and visual dialogue. Data on request. Dataset.
  • FoodieQA (EMNLP 2024 Main): Manually curated Chinese regional food benchmark: 389 unseen images, multi-image, single-image and text QA; open VLMs trail humans by 41% and 21%. Data gated. Dataset, Code.
  • K-Viscuit (ACL 2025 Main): Korean culture VQA benchmark of 657 questions on 237 images across ten categories, built with VLM-generated questions validated by native speakers. Dataset.

Middle East and North Africa

  • M2CQA (ACL 2026 Findings): Images from 17 MENA countries with true and counterfactual statements in English, Modern Standard Arabic, Egyptian and Levantine Arabic, for measuring counterfactual hallucination. Dataset.
  • PEARL (EMNLP 2025 Findings): Over 309K Arabic multimodal instruction examples across ten cultural domains covering all Arab countries, built with human-in-the-loop annotation by 37 annotators. Data gated. Dataset, Code, Project page.

Africa

  • Afri-MCQA (ACL 2026 Main): Culturally grounded multiple-choice and open-ended VQA in 15 African languages from 12 countries, about 7.5k QA pairs with native-language and accented-English audio. Dataset.

Visual reasoning, grounding and retrieval

Beyond recognition: visually grounded reasoning, grounding and segmentation of cultural objects, and culture-aware retrieval.

  • M4-RAG (CVPR 2026): Retrieval-augmented VQA benchmark of over 80,000 image-question pairs across 42 languages, 56 dialects and registers, and 189 countries, with a controlled multilingual document collection. Dataset, Code.
  • RAVENEA (ICLR 2026): 1,868 culture-focused instances with 11,396 human-ranked Wikipedia documents for retrieval-augmented cultural VQA and captioning across 8 countries. Dataset, Code, Project page.
  • GlobalRG (EMNLP 2024 Main): Retrieving culturally diverse images for 20 universal concepts across 50 countries, and grounding culture-specific concepts in images from 15 countries. Dataset, Code, Project page.
  • GD-VCR (EMNLP 2021 Main): 328 movie and TV screenshots with 886 four-way QA pairs over West, East Asia, South Asia and Africa, exposing regional performance gaps. Dataset.
  • MaRVL (EMNLP 2021 Main): 5,670 native-speaker true/false statements over image pairs in Indonesian, Chinese, Swahili, Tamil and Turkish, built from speaker-selected concepts. Dataset, Code, Project page.

Norms, values and safety

Whether models read social norms, values and offensiveness the way people in a given place do.

  • NormViz (COLM 2026): Contrastive image pairs from 16 countries test whether VLMs judge behaviors against local social norms; includes a 64k image training set with explanations. Data coming soon. Code.
  • CROSS (TMLR 2025): 1,284 image-grounded queries from 16 countries in 14 languages testing cultural norm safety; best model scores 61.79% awareness, 37.73% compliance. Dataset.
  • MC-SIGNS (ACL 2025 Main): 288 gesture-country pairs (25 gestures, 85 countries) annotated for offensiveness, used to test T2I systems, LLMs and VLMs for US-centric bias. Dataset.

Memes, humor, art and heritage

Memes and humor need shared context; art and heritage need domain knowledge. Both fail in culturally specific ways.

  • AVMeme Exam (arXiv, 2026): Human-curated benchmark of 1,032 iconic Internet audio-visual memes in more than ten languages, with questions from surface content to context, emotion, usage and world knowledge. Data gated. Dataset, Project page.
  • DuwatBench (EACL 2026 Main): 1,272 Arabic calligraphy samples with about 1,475 unique words across six styles and sentence-level detection annotations; 13 multimodal models evaluated. Dataset, Code, Project page.
  • MemeCULT-1K (EMNLP 2026 Main): 1,000 South Asian memes in Bengali, English and Hindi with context notes and human explanations; evaluates 13 VLMs with and without cultural context. Data coming soon. Code.
  • VULCA-Bench (arXiv, 2026): Art-critique benchmark of 7,410 image-critique pairs across 8 cultural traditions with bilingual Chinese-English expert critiques over a five-layer framework from perception to aesthetics. Dataset, Code.
  • TimeTravel (ACL 2025 Findings): 10,250 expert-verified samples of historical artifacts from 266 cultures across 10 historical regions, covering manuscripts, artworks, inscriptions and archaeology. Dataset, Code, Project page.
  • Multi3Hate (NAACL 2025 Main): 300 parallel memes in five languages annotated for hate speech by 445 annotators across five cultures; VLMs align more with US labels than with other cultures. Dataset, Code.

Captioning, translation and transcreation

Describing, translating and adapting images across languages and cultures.

  • CaMMT (EMNLP 2025 Findings): Over 5,800 image plus parallel English and regional-language caption triples from 23 regions for testing whether images help culturally aware translation. Dataset.
  • CIC (IJCAI 2024): Pipeline generating culture-category questions, extracting cultural elements via VQA, and prompting an LLM to write culturally descriptive captions, judged by 45 evaluators. Code, Project page.
  • MosAIC (NAACL 2025 Main): Multi-agent framework with cultural personas that writes culture-enriched captions for 2,832 images from China, India and Romania, with a culture-adaptable metric. Dataset.
  • Crossmodal-3600 (EMNLP 2022 Main): Geographically diverse image captioning evaluation set: 3,600 images with 261,375 human captions in 36 languages, 100 images per language region. Dataset.

Video understanding

Cultural understanding over time: events, rituals, norms and moments in video.

  • MINERVA-Cultural (CVPR 2026, as CURVE): Human-annotated long-video QA with reasoning traces in 18 native languages over 540 culturally specific videos from 18 locales; Video-LLMs fall well below human accuracy. Dataset.
  • VideoVista-CulturalLingo (ACL 2025 Main): Bilingual (Chinese, English) video QA benchmark with 1,389 videos and 3,134 questions spanning Chinese, North American and European cultural content; 24 models evaluated. Dataset, Code, Project page.
  • ViMUL-Bench (EMNLP 2025 Main): Video QA in 14 languages: 879 videos and 8,025 questions over 8 cultural and 7 generic categories, plus the ViMUL model. Dataset, Code, Project page.

Text-to-image and text-to-video generation

How faithfully generative models depict cultures, and how to measure and fix the gaps.

  • CultureVidBench (EMNLP 2026 Main): 1,000 prompts over 12 countries and 14 cultural aspects for judging cultural faithfulness of text-to-video models, with human and MLLM evaluation of seven models. Dataset, Project page.
  • When Cultures Move (EMNLP 2026 Workshop (NLP4PI)): 243 prompts and 972 generated videos across Chinese, American and Romanian cultures, mono and cross-cultural, plus MAVEN multi-agent prompt refinement for cultural fidelity. Dataset, Code.
  • CAIRE (EACL 2026 Main): Retrieval-augmented metric grounding image entities to a knowledge base to score cultural relevance per label, validated on rare-item and 10-country universal-concept sets. Dataset, Code.
  • CultDiff (ACL 2025 Main): Tests whether text-to-image diffusion models generate the architecture, clothing and food of 10 countries, with a learned similarity metric, CultDiff-S. No public data found.
  • CULTIVate (ICLR 2026): 576 social-activity prompts from 16 countries and over 19,000 generated images, scored for alignment, hallucination, exaggeration and diversity. Dataset, Code, Project page.
  • CulturalFrames (EMNLP 2025 Findings): Text-to-image benchmark of 983 prompts across 10 countries and 5 social domains, with 3,637 images and over 10,000 human annotations of cultural expectations. Dataset, Code, Project page.
  • Culture-TRIP (NAACL 2025 Main): Iterative prompt refinement that retrieves cultural context for culture nouns from 8 countries, improving Stable Diffusion alignment in a 66-participant study. Code, Project page.
  • CuRe (ICCV 2025): 300 cultural artifacts from 64 countries in 32 subcategories across six axes, scoring text-to-image systems on long-tail cultural fidelity. Dataset, Code, Project page.
  • Exposing Blindspots (IASEAI 2026): Audits text-to-image generation and image-to-image editing across six countries with an 8-category, 36-subcategory, era-aware prompt schema and native expert ratings. Dataset, Code, Project page.
  • When Cultures Meet (ACL 2026 Findings): Multicultural text-to-image benchmark of 9,000 generated images mixing people and landmarks from five countries in five languages, with the MosAIG multi-agent prompting framework. Dataset, Code.
  • Where Culture Fades (CVPR 2026): Shows multilingual T2I models give culturally neutral or English-biased images across 15 languages, localizes culture neurons, and proposes activation and layer-targeted fixes on CultureBench.
  • CUBE (NeurIPS 2024 Datasets and Benchmarks): 1,000 prompts and about 300,000 cultural artifacts across cuisine, landmarks and art in 8 countries, for text-to-image cultural awareness and diversity. Dataset.
  • Image transcreation (EMNLP 2024 Main): Three generative pipelines and 700 evaluation images across 7 countries for adapting images to a target culture; the best pipelines translate only 5% of images for some countries. Dataset, Code, Project page.
  • SCoFT (CVPR 2024): Self-contrastive fine-tuning of Stable Diffusion on the CCUB dataset, judged by 51 participants from 5 countries to reduce stereotypes and raise cultural relevance. Dataset, Code, Project page.
  • ViSAGe (ACL 2024 Main): Visual stereotype attributes for 135 nationalities; stereotypical attributes are three times as likely in T2I images of those identities, most for Global South groups. Dataset.
  • CCUB (arXiv, 2023): 1,095 image-text pairs from 8 countries, curated by people with ties to each culture, for fine-tuning Stable Diffusion and prompting GPT-3 toward culturally relevant generation. Dataset.
  • CulText2I (TACL 2025): Images from six TTI models prompted in ten languages, evaluated with CLIP, VQA and humans to expose each model’s cultural point of view. Data on request. Dataset, Project page.
  • DIG In (TMLR 2023): Three automatic indicators of realism, diversity and prompt-generation consistency for T2I images of objects worldwide; generations for Africa and West Asia score lower than Europe. Dataset, Project page.
  • Geographical representativeness of T2I (ICCV 2023): 540 participants from 27 countries rate whether DALL-E 2 and Stable Diffusion images of common nouns reflect their country; defaults skew to the US.

Geographic robustness of visual recognition

The older and more basic question: does image recognition work everywhere, or only where the training data came from?

  • Multilingual diversity improves VL representations (NeurIPS 2024 Main): Shows that translating non-English web captions to English and re-filtering improves vision-language pretraining, with gains on GeoDE across regions, largest in Africa. Dataset, Project page.
  • No Filter (NeurIPS 2024 Main): Filtering pre-training data to English image-text pairs hurts cultural understanding and lower-income communities; global pre-training before English fine-tuning improves it without losing benchmark accuracy.
  • Does progress on object recognition benchmarks improve generalization? (ICLR 2024): Evaluates nearly 100 vision models on DollarStreet and GeoDE, finding 7 to 20 percent geographic accuracy gaps that ImageNet progress does not close. Project page.
  • GeoDE (NeurIPS 2023 Datasets and Benchmarks): 61,940 crowdsourced images of 40 object classes from six world regions, without PII, for geographically diverse evaluation and training. Dataset, Code.
  • GeoNet (CVPR 2023): Domain adaptation benchmark with USA and Asia splits for scene recognition (205 classes) and object classification (600 classes), plus a universal adaptation split. Dataset, Code.
  • Dollar Street (NeurIPS 2022 Datasets and Benchmarks): Supervised dataset of 38,479 household-item images from homes worldwide, labeled with object tags, region, country and monthly income, released under CC-BY. Dataset, Project page.
  • Does Object Recognition Work for Everyone? (CVPR 2019 Workshops): Evaluates five commercial object-recognition APIs on Dollar Street household photos from 54 countries, finding lower accuracy for low-income homes. Dataset.

Models, training data and adaptation methods

Training data, pre-training objectives, adaptation and prompting methods that target cultural or geographic gaps.

  • CulturalGround (EMNLP 2025 Main): 22M synthetic multilingual VQA pairs on images of culturally significant Wikidata entities from 42 countries in 39 languages, used to train CulturalPangea-7B. Dataset, Code, Project page.
  • CultureCLIP (COLM 2025): Fine-tunes CLIP on CulTwin, a synthetic set of 73,823 retained concept-caption-image triplets spanning 229 countries, to sharpen fine-grained cultural concept recognition. Code.
  • CultureMixup (EMNLP 2023 Main): Annotation-free cultural concept mapping plus a mixup augmentation that improve four multilingual VLMs on MaRVL across five languages. Dataset, Code.
  • GIVL (CVPR 2023): Pre-trained VLM with Image-Knowledge Matching and Image Edit Checking objectives to reduce performance gaps on geo-diverse benchmarks such as GD-VCR and MaRVL. Code.

Analyses, position papers and surveys

Where the failures come from, and how the field should evaluate.

  • Cultural representation disparities (IJCNLP-AACL 2025 Findings): Probes VLMs on country identification over Country211 (211 countries) with open-ended, multiple-choice, multilingual and adversarial settings; prompts released on Hugging Face. Dataset.
  • Cultural theory for VLM evaluation (arXiv, 2025): Position paper arguing VLM cultural evaluations should draw on cultural studies, semiotics and visual studies, proposing five frameworks after reviewing 35 recent papers.
  • Culture-sensitive neurons (EACL 2026 Main): Identifies culture-sensitive neurons in three VLMs on CVQA’s 25 cultural groups, with a contrastive activation margin method validated by ablation. Code.
  • See It from My Perspective (ICLR 2025): Analysis showing VLMs favor Western over East Asian image subsets and that pre-training language mix and inference language shift the bias. Code, Project page.

Foundational reading

Background that motivates the whole area.

  • The weirdest people in the world? (Behavioral and Brain Sciences, 2010): Psychology review arguing that findings from Western, Educated, Industrialized, Rich and Democratic samples are outliers and do not generalize to humanity.