WorldCuisines: A Massive-Scale Benchmark for Multilingual and Multicultural Visual Question Answering on Global Cuisines
Winata, Genta Indra, Hudi, Frederikus, Irawan, Patrick Amadeus, Anugraha, David, Putri, Rifki Afina, Wang, Yutong, Nohejl, Adam, Prathama, Ubaidillah Ariq, Ousidhoum, Nedjma, Amriani, Afifa, Rzayev, Anar, Das, Anirban, Pramodya, Ashmari, Adila, Aulia, Wilie, Bryan, Mawalim, Candy Olivia, Cheng, Ching Lam, Abolade, Daud, Chersoni, Emmanuele, Santus, Enrico, Ikhwantri, Fariz, Kuwanto, Garry, Zhao, Hanyang, Wibowo, Haryo Akbarianto, Lovenia, Holy, Cruz, Jan Christian Blaise, Putra, Jan Wira Gotama, Myung, Junho, Susanto, Lucky, Machin, Maria Angelica Riera, Zhukova, Marina, Anugraha, Michael, Adilazuarda, Muhammad Farid, Santosa, Natasha, Limkonchotiwat, Peerat, Dabre, Raj, Audino, Rio Alexander, Cahyawijaya, Samuel, Zhang, Shi-Xiong, Salim, Stephanie Yulia, Zhou, Yi, Gui, Yinxuan, Adelani, David Ifeoluwa, Lee, En-Shiun Annie, Okada, Shogo, Purwarianti, Ayu, Aji, Alham Fikri, Watanabe, Taro, Wijaya, Derry Tanti, Oh, Alice, Ngo, Chong-Wah
–arXiv.org Artificial Intelligence
Vision Language Models (VLMs) often struggle with culture-specific knowledge, particularly in languages other than English and in underrepresented cultural contexts. To evaluate their understanding of such knowledge, we introduce WorldCuisines, a massive-scale benchmark for multilingual and multicultural, visually grounded language understanding. This benchmark includes a visual question answering (VQA) dataset with text-image pairs across 30 languages and dialects, spanning 9 language families and featuring over 1 million data points, making it the largest multicultural VQA benchmark to date. It includes tasks for identifying dish names and their origins. We provide evaluation datasets in two sizes (12k and 60k instances) alongside a training dataset (1 million instances). Our findings show that while VLMs perform better with correct location context, they struggle with adversarial contexts and predicting specific regional cuisines and languages. To support future research, we release a knowledge base with annotated food entries and images along with the VQA data.
arXiv.org Artificial Intelligence
Nov-28-2024
- Country:
- South America
- Oceania
- New Zealand (0.04)
- Australia (0.04)
- Micronesia (0.04)
- Fiji (0.04)
- North America
- United States (0.14)
- Trinidad and Tobago (0.04)
- Costa Rica (0.04)
- Puerto Rico (0.04)
- Dominican Republic (0.04)
- Antigua and Barbuda (0.04)
- Honduras (0.04)
- Nicaragua (0.04)
- Guatemala (0.04)
- Anguilla (0.04)
- Dominica (0.04)
- Barbados (0.04)
- Saint Kitts and Nevis (0.04)
- Mexico (0.04)
- Jamaica (0.04)
- El Salvador (0.04)
- Central America (0.04)
- Saint Lucia (0.04)
- The Bahamas (0.04)
- Haiti (0.04)
- Cuba (0.04)
- Saint Vincent and the Grenadines (0.04)
- Canada > Ontario (0.04)
- Europe
- North Macedonia (0.04)
- Montenegro (0.04)
- Faroe Islands (0.04)
- Bosnia and Herzegovina (0.04)
- Austria (0.04)
- Bulgaria (0.04)
- Liechtenstein (0.04)
- Poland (0.04)
- Germany (0.04)
- Spain (0.04)
- Netherlands (0.04)
- Iceland (0.04)
- Switzerland (0.04)
- Denmark (0.04)
- Finland (0.04)
- Albania (0.04)
- Monaco (0.04)
- Norway (0.04)
- Slovakia (0.04)
- Slovenia (0.04)
- Serbia (0.04)
- France (0.04)
- Russia (0.04)
- Greece (0.04)
- Latvia (0.04)
- Western Europe (0.04)
- United Kingdom > Wales (0.04)
- Lithuania (0.04)
- Romania (0.04)
- Eastern Europe (0.04)
- Belgium (0.04)
- Ukraine (0.04)
- Czechia (0.04)
- Ireland (0.04)
- Northern Europe (0.04)
- Moldova (0.04)
- Portugal (0.04)
- Hungary (0.04)
- Middle East
- Italy > Calabria
- Catanzaro Province > Catanzaro (0.04)
- Asia
- Brunei (0.14)
- Singapore (0.04)
- China > Hong Kong (0.04)
- Afghanistan (0.04)
- India (0.04)
- Tajikistan (0.04)
- Thailand (0.04)
- Turkmenistan (0.04)
- Sri Lanka (0.04)
- Kyrgyzstan (0.04)
- Myanmar (0.04)
- Taiwan (0.04)
- South Korea (0.04)
- Armenia (0.04)
- Timor-Leste (0.04)
- Russia (0.04)
- Philippines (0.04)
- Maldives (0.04)
- Laos (0.04)
- Vietnam (0.04)
- Azerbaijan (0.04)
- Bhutan (0.04)
- Bangladesh (0.04)
- Uzbekistan (0.04)
- Central Asia (0.04)
- Kazakhstan (0.04)
- North Korea (0.04)
- Macao (0.04)
- Malaysia (0.04)
- Mongolia (0.04)
- Nepal (0.04)
- Cambodia (0.04)
- Pakistan (0.04)
- Middle East
- Indonesia
- Java > Central Java (0.04)
- Bali (0.04)
- Japan > Honshū
- Kantō > Tokyo Metropolis Prefecture > Tokyo (0.14)
- Africa
- Niger (0.04)
- Zambia (0.04)
- Benin (0.04)
- Comoros (0.04)
- Mauritius (0.04)
- Burundi (0.04)
- Cameroon (0.04)
- Senegal (0.04)
- Angola (0.04)
- Liberia (0.04)
- Madagascar (0.04)
- Western Sahara (0.04)
- East Africa (0.04)
- Uganda (0.04)
- Mozambique (0.04)
- Rwanda (0.04)
- Mauritania (0.04)
- Seychelles (0.04)
- Eritrea (0.04)
- Zimbabwe (0.04)
- Namibia (0.04)
- Ghana (0.04)
- Central African Republic (0.04)
- Nigeria (0.04)
- The Gambia (0.04)
- West Africa (0.04)
- South Africa (0.04)
- Cabo Verde (0.04)
- Sudan (0.04)
- Ethiopia (0.04)
- Burkina Faso (0.04)
- Sierra Leone (0.04)
- Southern Africa (0.04)
- Lesotho (0.04)
- North Africa (0.04)
- Togo (0.04)
- Middle East
- Genre:
- Research Report > New Finding (0.68)
- Technology:
- Information Technology > Artificial Intelligence
- Machine Learning (1.00)
- Natural Language
- Question Answering (0.70)
- Large Language Model (0.70)
- Chatbot (0.48)
- Information Technology > Artificial Intelligence