Thought-For-Food: Reasoning Chain Induced Food Visual Question Answering

Abstract--The immense diversity in the culture and culinary of Indian cuisines calls attention to the major shortcoming of the existing Visual Question Answering(VQA) systems which are inclined towards the foods from western regionRecent attempt towards building a VQA dataset for Indian food is a step towards addressing this challenge. However, their approach towards VQA follows a two-step process in which the answer is generated first, followed by the explanation of the expected answer . In this work, we claim that food VQA requires to follow a multi-step reasoning process to arrive at an accurate answer, especially in the context of India food, which involves understanding complex culinary context and identifying relationships between various food items. With this hypothesis we create reasoning chains upon the QA with minimal human intervention. With augmentation of reasoning chains, we observed accuracy improvement of an average 10 percentage points on the baseline. We provide detailed analysis in terms the effect of addition of reasoning chains for the Indian Food VQA task. One of the most important part of culture and social aspects in everyday life is food. In a country like India, food highlights immense diversity based on geography, religion, and traditions of different regions. A single mealcontain items which differ in preparation, presentation and flavor. This richness in the culinary and the culture, poses unique set of challenges for AI systems that target the understanding of content related to Indian food. A powerful framework that has emerged to connect visual and language reasoning is Visual Question Answering(VQA) [6].