Goto

Collaborating Authors

 odia


BHRAM-IL: A Benchmark for Hallucination Recognition and Assessment in Multiple Indian Languages

Terdalkar, Hrishikesh, Bhojani, Kirtan, Dongare, Aryan, Behera, Omm Aditya

arXiv.org Artificial Intelligence

Large language models (LLMs) are increasingly deployed in multilingual applications but often generate plausible yet incorrect or misleading outputs, known as hallucinations. While hallucination detection has been studied extensively in English, under-resourced Indian languages remain largely unexplored. We present BHRAM-IL, a benchmark for hallucination recognition and assessment in multiple Indian languages, covering Hindi, Gujarati, Marathi, Odia, along with English. The benchmark comprises 36,047 curated questions across nine categories spanning factual, numerical, reasoning, and linguistic tasks. We evaluate 14 state-of-the-art multilingual LLMs on a benchmark subset of 10,265 questions, analyzing cross-lingual and factual hallucinations across languages, models, scales, categories, and domains using category-specific metrics normalized to (0,1) range. Aggregation over all categories and models yields a primary score of 0.23 and a language-corrected fuzzy score of 0.385, demonstrating the usefulness of BHRAM-IL for hallucination-focused evaluation. The dataset, and the code for generation and evaluation are available on GitHub (https://github.com/sambhashana/BHRAM-IL/) and HuggingFace (https://huggingface.co/datasets/sambhashana/BHRAM-IL/) to support future research in multilingual hallucination detection and mitigation.


Global virtual conference Odias in ML to be held on October 4

#artificialintelligence

A global virtual conference aimed at promoting the use of artificial intelligence (AI) and machine learning (ML) for the development of Odisha and advancement of Odia language in the digital era is being organized on 4th October Sunday, on the virtual meeting platform Zoom. The conference, called Odias in ML Conference, is being organized by a group of Odias, also called Odias in ML, with a shared interest in AI and ML. The conference also aims to showcase career and entrepreneurship opportunities in AI and ML for Odias across the world. This first-of-its-kind conference will see participation of a multitude of stakeholders including technologists, researchers, academicians, business executives, entrepreneurs, policymakers, linguists, language activists, media persons and community leaders, all with a commitment to AI and machine learning. The speakers, all Odias based across three continents, will come together to brainstorm how the opportunities created by these emerging technologies can be leveraged effectively to propel the next phase of growth for Odisha and Odias.


"Odias in Machine Learning" Conference

#artificialintelligence

This auspicious Durga Pooja season 2020 we are organizing the First Online Edition of the Odias in Machine Learning Conference. ଏବର୍ଷ ପବିତ୍ର ଦୁର୍ଗାପୂଜାରେ "ଯନ୍ତ୍ର-ସ୍ୱୟଂଶିକ୍ଷା ବିଷୟରେ ଓଡ଼ିଆ ମାନେ" ଶୀର୍ଷକରେ ଏକ ଅନ-ଲାଇନ କନଫରେନ୍ସ ଆୟୋଜନ କରାଯାଉଛି। The aim is to have a virtual conference of Odias in the field of AI/ML practice and policy to come together and discuss future prospects and apprehensions. କୃତ୍ରିମ ବୁଦ୍ଧିମତା (ଏ ଆଇ/AI) / ଯନ୍ତ୍ର-ସ୍ୱୟଂଶିକ୍ଷା (ଏମ୍ ଏଲ୍/ML) ରେ କାର୍ଯ୍ୟରତ, ଗବେଷଣାରତ ଓଡ଼ିଆ ମାନଙ୍କୁ ଏକତ୍ରୀକରଣ କରି ସମ୍ମିଳିତ ଭାବରେ ଭବିଷ୍ୟତର ସମ୍ଭାବନା, କାର୍ଯ୍ୟପନ୍ଥା ଓ ନୀତି ନିର୍ଧାରଣତିଆରି କରିବା ଏହି ସମ୍ମିଳନୀର ଲକ୍ଷ॥