Towards Large Language Model driven Reference-less Translation Evaluation for English and Indian Languages

With the primary focus on evaluating the effectiveness of large language models for automatic reference-less translation assessment, this work presents our experiments on mimicking human direct assessment to evaluate the quality of translations in English and Indian languages. We constructed a translation evaluation task where we performed zero-shot learning, in-context example-driven learning, and fine-tuning of large language models to provide a score out of 100, where 100 represents Figure 1: Spearman co-relation: Human translation a perfect translation and 1 represents a poor evaluation vs different reference-less translation translation. We compared the performance of evaluation metrics. Llama-2-7b-Adapt (lora), our trained systems with existing methods such Llama-2-13b-Adapt (lora), Mistral-7b-Adpt (lora), as COMET, BERT-Scorer, and LABSE, and COMET-QE (https://github.com/Unbabel/COMET)