Introducing ABENA: BERT Natural Language Processing for Twi

In our previous blog post we introduced a preliminary Twi embedding model based on fastText and visualized it using the Tensorflow Embedding Projector. As a reminder, text embeddings allow you to convert text into numbers or vectors which a computer can perform arithmetic operations on to enable it reason about human language, i.e., carry out natural language processing (NLP). A screenshot of our fastText Twi embeddings from that exercise is shown in Figure 1. This model-- which we have shared in our Kasa Library repo -- enables a computer to begin to reason in Twi computationally. However it is "static" in the sense that the vectors do not change with different contexts. State-of-the-art NLP in high-resource languages such as English has largely moved away from these to more sophisticated "dynamic" embeddings capable of understanding a changing contexts.