Dynamic Word Tokenization with Regex Tokenizer

In the realm of machine learning engineering, scoping, data handling, modelling and deployment are the main recursive processes of a project lifecycle. Of note, data cleaning and preparation are considered the early stages of any data science project pipeline, yet they are of paramount importance to the model's accuracy. For structured tabular data, data preprocessing may take the form of imputation of missing data or standardizing values of certain classes (e.g. Yet for this tutorial, we will be touching on data preprocessing method on unstructured data from another sub-field, Natural Language Processing (NLP) -- text data. If images (another unstructured data) are considered spatial data, then text should be considered sequential data, with information of text being derived, after tokens (words or characters) are processed in complete order.