WorldCuisines: A Massive-Scale Benchmark for Multilingual and Multicultural Visual Question Answering on Global Cuisines
Winata, Genta Indra, Hudi, Frederikus, Irawan, Patrick Amadeus, Anugraha, David, Putri, Rifki Afina, Wang, Yutong, Nohejl, Adam, Prathama, Ubaidillah Ariq, Ousidhoum, Nedjma, Amriani, Afifa, Rzayev, Anar, Das, Anirban, Pramodya, Ashmari, Adila, Aulia, Wilie, Bryan, Mawalim, Candy Olivia, Cheng, Ching Lam, Abolade, Daud, Chersoni, Emmanuele, Santus, Enrico, Ikhwantri, Fariz, Kuwanto, Garry, Zhao, Hanyang, Wibowo, Haryo Akbarianto, Lovenia, Holy, Cruz, Jan Christian Blaise, Putra, Jan Wira Gotama, Myung, Junho, Susanto, Lucky, Machin, Maria Angelica Riera, Zhukova, Marina, Anugraha, Michael, Adilazuarda, Muhammad Farid, Santosa, Natasha, Limkonchotiwat, Peerat, Dabre, Raj, Audino, Rio Alexander, Cahyawijaya, Samuel, Zhang, Shi-Xiong, Salim, Stephanie Yulia, Zhou, Yi, Gui, Yinxuan, Adelani, David Ifeoluwa, Lee, En-Shiun Annie, Okada, Shogo, Purwarianti, Ayu, Aji, Alham Fikri, Watanabe, Taro, Wijaya, Derry Tanti, Oh, Alice, Ngo, Chong-Wah
–arXiv.org Artificial Intelligence
Vision Language Models (VLMs) often struggle with culture-specific knowledge, particularly in languages other than English and in underrepresented cultural contexts. To evaluate their understanding of such knowledge, we introduce WorldCuisines, a massive-scale benchmark for multilingual and multicultural, visually grounded language understanding. This benchmark includes a visual question answering (VQA) dataset with text-image pairs across 30 languages and dialects, spanning 9 language families and featuring over 1 million data points, making it the largest multicultural VQA benchmark to date. It includes tasks for identifying dish names and their origins. We provide evaluation datasets in two sizes (12k and 60k instances) alongside a training dataset (1 million instances). Our findings show that while VLMs perform better with correct location context, they struggle with adversarial contexts and predicting specific regional cuisines and languages. To support future research, we release a knowledge base with annotated food entries and images along with the VQA data.
arXiv.org Artificial Intelligence
Nov-28-2024
- Country:
- Africa
- Togo (0.04)
- North Africa (0.04)
- Lesotho (0.04)
- Southern Africa (0.04)
- Sierra Leone (0.04)
- Burkina Faso (0.04)
- Ethiopia (0.04)
- Niger (0.04)
- Sudan (0.04)
- Cabo Verde (0.04)
- South Africa (0.04)
- West Africa (0.04)
- The Gambia (0.04)
- Nigeria (0.04)
- Central African Republic (0.04)
- Ghana (0.04)
- Namibia (0.04)
- Middle East
- Zimbabwe (0.04)
- Eritrea (0.04)
- Seychelles (0.04)
- Mauritania (0.04)
- Rwanda (0.04)
- Mozambique (0.04)
- Uganda (0.04)
- East Africa (0.04)
- Western Sahara (0.04)
- Madagascar (0.04)
- Liberia (0.04)
- Angola (0.04)
- Senegal (0.04)
- Cameroon (0.04)
- Burundi (0.04)
- Mauritius (0.04)
- Comoros (0.04)
- Benin (0.04)
- Zambia (0.04)
- Asia
- Pakistan (0.04)
- Japan > Honshū
- Kantō > Tokyo Metropolis Prefecture > Tokyo (0.14)
- Brunei (0.14)
- Cambodia (0.04)
- Nepal (0.04)
- Mongolia (0.04)
- Malaysia (0.04)
- Macao (0.04)
- North Korea (0.04)
- Kazakhstan (0.04)
- Central Asia (0.04)
- Uzbekistan (0.04)
- Bangladesh (0.04)
- Bhutan (0.04)
- Azerbaijan (0.04)
- Vietnam (0.04)
- Indonesia
- Bali (0.04)
- Java > Central Java (0.04)
- Laos (0.04)
- Middle East
- Maldives (0.04)
- Philippines (0.04)
- Russia (0.04)
- Timor-Leste (0.04)
- Armenia (0.04)
- South Korea (0.04)
- Taiwan (0.04)
- Myanmar (0.04)
- Kyrgyzstan (0.04)
- Sri Lanka (0.04)
- Turkmenistan (0.04)
- Thailand (0.04)
- Singapore (0.04)
- Tajikistan (0.04)
- India (0.04)
- Afghanistan (0.04)
- China > Hong Kong (0.04)
- Europe
- North Macedonia (0.04)
- Hungary (0.04)
- Portugal (0.04)
- Moldova (0.04)
- Northern Europe (0.04)
- Ireland (0.04)
- Czechia (0.04)
- Ukraine (0.04)
- Belgium (0.04)
- Eastern Europe (0.04)
- Romania (0.04)
- Italy > Calabria
- Catanzaro Province > Catanzaro (0.04)
- Lithuania (0.04)
- United Kingdom > Wales (0.04)
- Western Europe (0.04)
- Latvia (0.04)
- Middle East
- Greece (0.04)
- Russia (0.04)
- France (0.04)
- Serbia (0.04)
- Slovenia (0.04)
- Slovakia (0.04)
- Norway (0.04)
- Monaco (0.04)
- Albania (0.04)
- Finland (0.04)
- Denmark (0.04)
- Switzerland (0.04)
- Iceland (0.04)
- Netherlands (0.04)
- Spain (0.04)
- Germany (0.04)
- Poland (0.04)
- Liechtenstein (0.04)
- Bulgaria (0.04)
- Austria (0.04)
- Bosnia and Herzegovina (0.04)
- Faroe Islands (0.04)
- Montenegro (0.04)
- North America
- Canada > Ontario (0.04)
- Saint Vincent and the Grenadines (0.04)
- Cuba (0.04)
- Haiti (0.04)
- The Bahamas (0.04)
- Saint Lucia (0.04)
- Central America (0.04)
- El Salvador (0.04)
- United States (0.14)
- Jamaica (0.04)
- Mexico (0.04)
- Saint Kitts and Nevis (0.04)
- Barbados (0.04)
- Dominica (0.04)
- Anguilla (0.04)
- Guatemala (0.04)
- Nicaragua (0.04)
- Honduras (0.04)
- Antigua and Barbuda (0.04)
- Dominican Republic (0.04)
- Puerto Rico (0.04)
- Costa Rica (0.04)
- Trinidad and Tobago (0.04)
- Oceania
- Australia (0.04)
- Fiji (0.04)
- Micronesia (0.04)
- New Zealand (0.04)
- South America
- Africa
- Genre:
- Research Report > New Finding (0.68)
- Technology:
- Information Technology > Artificial Intelligence
- Machine Learning (1.00)
- Natural Language
- Chatbot (0.48)
- Large Language Model (0.70)
- Question Answering (0.70)
- Information Technology > Artificial Intelligence