Consonant-Vowel Sequences as Subword Units for Code-Mixed Languages

They used character n-grams as sub-word units that were obtained The evolution of social media texts such as blogs, microblogs as convolutions over characters and passed to a LSTM layer (e.g., Twitter), WhatsApp, and informal chats have followed by softmax layer. For Hi-En code-mixed text (Joshi created many new opportunities for information access and et al. 2016) address the problem of rare or out-of-vocabulary language technologies, but have also presented many new words without any text normalization. In this paper, we propose challenges. This makes it one of the primary research areas a novel approach, without any need of explicit text normalization, of the present era. In social media, non-English speakers for creating sub-word units and a new hierarchical [according to statistics half of messages on Twitter arent model that efficiently learns sentence representations in English (Schroeder, Minocha, and Schneider 2010)] from these units.