Word Segmentation for Asian Languages: Chinese, Korean, and Japanese

Thus, word segmentation is important and is influential in many fields including developing text processing applications, such as Information Extraction, Document Summarization, Machine Translation (MT), Natural Language Processing, Information Retrieval, Language Modeling, and Speech Recognition.(15) Word segmentation is often a vital task of language processing. In addition, the reason why word segmentation is significant in the field of Natural Language Processing is because it is the initial step for most higher level natural language processing tasks, such as part-of-speech tagging and parsing. In addition, for languages that are space-delimited such as English or Russian, these languages are being segmented differently as opposed to those that don't have explicit word boundary delimiters, such as Chinese and Japanese. There is a common goal for this task, which is to have a near-perfect word segmentation system, which can still perform reasonably with no or minimum language-specific adaptations (9).