Enhancing Neural Machine Translation of Low-Resource Languages: Corpus Development, Human Evaluation and Explainable AI Architectures
–arXiv.org Artificial Intelligence
In the current machine translation (MT) landscape, the Transformer architecture stands out as the gold standard, especially for high-resource language pairs. This research delves into its efficacy for low-resource language pairs including both the English$\leftrightarrow$Irish and English$\leftrightarrow$Marathi language pairs. Notably, the study identifies the optimal hyperparameters and subword model type to significantly improve the translation quality of Transformer models for low-resource language pairs. The scarcity of parallel datasets for low-resource languages can hinder MT development. To address this, gaHealth was developed, the first bilingual corpus of health data for the Irish language. Focusing on the health domain, models developed using this in-domain dataset exhibited very significant improvements in BLEU score when compared with models from the LoResMT2021 Shared Task. A subsequent human evaluation using the multidimensional quality metrics error taxonomy showcased the superior performance of the Transformer system in reducing both accuracy and fluency errors compared to an RNN-based counterpart. Furthermore, this thesis introduces adaptNMT and adaptMLLM, two open-source applications streamlined for the development, fine-tuning, and deployment of neural machine translation models. These tools considerably simplify the setup and evaluation process, making MT more accessible to both developers and translators. Notably, adaptNMT, grounded in the OpenNMT ecosystem, promotes eco-friendly natural language processing research by highlighting the environmental footprint of model development. Fine-tuning of MLLMs by adaptMLLM demonstrated advancements in translation performance for two low-resource language pairs: English$\leftrightarrow$Irish and English$\leftrightarrow$Marathi, compared to baselines from the LoResMT2021 Shared Task.
arXiv.org Artificial Intelligence
Mar-3-2024
- Country:
- Oceania > Australia
- North America
- Dominican Republic (0.04)
- United States
- New York (0.04)
- Maryland > Baltimore (0.04)
- Texas > Travis County
- Austin (0.04)
- Pennsylvania > Philadelphia County
- Philadelphia (0.04)
- New Mexico > Santa Fe County
- Santa Fe (0.04)
- Minnesota > Hennepin County
- Minneapolis (0.14)
- Massachusetts
- Suffolk County > Boston (0.04)
- Middlesex County > Cambridge (0.04)
- California
- Santa Clara County > Palo Alto (0.04)
- San Diego County > San Diego (0.04)
- Alameda County > Berkeley (0.04)
- Canada
- Europe
- Germany > Berlin (0.04)
- Czechia > Prague (0.04)
- Iceland > Capital Region
- Reykjavik (0.04)
- Italy > Tuscany
- Florence (0.04)
- France > Provence-Alpes-Côte d'Azur
- Bouches-du-Rhône > Marseille (0.04)
- Denmark > Capital Region
- Copenhagen (0.04)
- Finland
- Portugal > Lisbon
- Lisbon (0.14)
- Spain > Catalonia
- Barcelona Province > Barcelona (0.04)
- Netherlands > South Holland
- Dordrecht (0.04)
- Middle East > Republic of Türkiye
- Istanbul Province > Istanbul (0.04)
- Croatia > Dubrovnik-Neretva County
- Dubrovnik (0.04)
- Ireland > Leinster
- County Dublin > Dublin (0.04)
- United Kingdom
- Northern Ireland (0.04)
- England > Greater London
- London (0.04)
- Belgium > Brussels-Capital Region
- Brussels (0.04)
- Asia
- Singapore (0.04)
- Macao (0.04)
- India > Maharashtra (0.04)
- China > Hong Kong (0.04)
- Middle East
- Republic of Türkiye > Istanbul Province
- Istanbul (0.04)
- Qatar > Ad-Dawhah
- Doha (0.04)
- Republic of Türkiye > Istanbul Province
- Japan > Honshū
- Kansai > Kyoto Prefecture > Kyoto (0.04)
- Africa > Ethiopia
- Addis Ababa > Addis Ababa (0.04)
- Genre:
- Summary/Review (1.00)
- Overview (1.00)
- Instructional Material (1.00)
- Research Report
- New Finding (1.00)
- Experimental Study (0.93)
- Industry:
- Information Technology > Services (1.00)
- Energy (1.00)
- Education (1.00)
- Health & Medicine
- Epidemiology (0.67)
- Therapeutic Area
- Infections and Infectious Diseases (1.00)
- Immunology (1.00)
- Government > Regional Government
- Europe Government (0.67)
- Technology: