YAD: Leveraging T5 for Improved Automatic Diacritization of Yor\`ub\'a Text

In addition, we pre-train text-to-text transformer, T5 model for Yorùbá and showed that this model outperform several multilingually trained T5 models. Lastly, we showed that more data and larger models are better at diacritization for Yorùbá Introduction Yorùbá, a language spoken predominantly in West Africa, is renowned for its tonal nature which is characterized by a heavy use of diacritics to signify tone variations. In Yorùbá and many other languages, diacritics play a crucial role in disambiguating word meanings and in word pronunciation, making accurate diacritization essential for effective communication and language processing tasks (Skiredj & Berrada, 2024). However, manual diacritization is time-consuming and requires specialized linguistic expertise, motivating the development of automatic diacritization systems. In recent years, significant progress has been made in natural language processing (NLP) techniques, leading to the exploration of various approaches to automate the diacritization process for languages using diacritics (Náplava et al., 2018; Mubarak et al., 2019; Náplava et al., 2021; Stankevicius et al., 2022, inter alia) including Yorùbá (Orife, 2018; Orife et al., 2020).