Instructional Material
Text Analysis and Natural Language Processing With Python - blackfree
My course provides a foundation to carry out PRACTICAL, real-life social media mining. By taking this course, you are taking an important step forward in your data science journey to become an expert in harnessing the power of social media for deriving insights and identifying trends. Why Should You Take My Course? I have an MPhil (Geography and Environment) from the University of Oxford, UK. I also completed a data science intense PhD at Cambridge University (Tropical Ecology and Conservation).
NLP Foundations - blackfree
Let's understand NLP and get all fundamental skills from SCRATCH! In this course you are invited to learn all the fundamental skills ... In this course you are invited to learn all the fundamental skills required in any kind of activity related to the Natural Language Processing and you will learn them from a theoretical and practical point of view, in fact you will seat together with me coding and implementing any topic step-by-step, instruction after instruction. Any of these projects will be a real and working use case so you will be able to re-use them in your own apps. In few words, this course is a real journey inside Natural Language Processing starting from the very beginning and finishing with the idea that all modern systems are leveraging: word embeddings. We are exploring NLU, NLG, NLP History, applications and use cases, studing Tokenization, Stopwords, Stemming, Lemmatization, PoS, NER, BoW, TF-IDF and Embeddings.
Machine Learning: Natural Language Processing in Python (V2) - blackfree
Welcome to Machine Learning: Natural Language Processing in Python (Version 2). In part 1, which covers vector models and text preprocessing methods, you will learn about why vectors are so essential in data science and artificial intelligence. You will learn about various techniques for converting text into vectors, such as the CountVectorizer and TF-IDF, and you'll learn the basics of neural embedding methods like word2vec, and GloVe. Along the way, you'll also learn important text preprocessing steps, such as tokenization, stemming, and lemmatization. You'll be introduced briefly to classic NLP tasks such as parts-of-speech tagging.
Genetic Algorithm for Program Synthesis
A deductive program synthesis tool takes a specification as input and derives a program that satisfies the specification. The drawback of this approach is that search spaces for such correct programs tend to be enormous, making it difficult to derive correct programs within a realistic timeout. To speed up such program derivation, we improve the search strategy of a deductive program synthesis tool, SuSLik, using evolutionary computation. Our cross-validation shows that the improvement brought by evolutionary computation generalises to unforeseen problems.
A Domain-Agnostic Approach for Characterization of Lifelong Learning Systems
Baker, Megan M., New, Alexander, Aguilar-Simon, Mario, Al-Halah, Ziad, Arnold, Sรฉbastien M. R., Ben-Iwhiwhu, Ese, Brna, Andrew P., Brooks, Ethan, Brown, Ryan C., Daniels, Zachary, Daram, Anurag, Delattre, Fabien, Dellana, Ryan, Eaton, Eric, Fu, Haotian, Grauman, Kristen, Hostetler, Jesse, Iqbal, Shariq, Kent, Cassandra, Ketz, Nicholas, Kolouri, Soheil, Konidaris, George, Kudithipudi, Dhireesha, Learned-Miller, Erik, Lee, Seungwon, Littman, Michael L., Madireddy, Sandeep, Mendez, Jorge A., Nguyen, Eric Q., Piatko, Christine D., Pilly, Praveen K., Raghavan, Aswin, Rahman, Abrar, Ramakrishnan, Santhosh Kumar, Ratzlaff, Neale, Soltoggio, Andrea, Stone, Peter, Sur, Indranil, Tang, Zhipeng, Tiwari, Saket, Vedder, Kyle, Wang, Felix, Xu, Zifan, Yanguas-Gil, Angel, Yedidsion, Harel, Yu, Shangqun, Vallabha, Gautam K.
Despite the advancement of machine learning techniques in recent years, state-of-the-art systems lack robustness to "real world" events, where the input distributions and tasks encountered by the deployed systems will not be limited to the original training context, and systems will instead need to adapt to novel distributions and tasks while deployed. This critical gap may be addressed through the development of "Lifelong Learning" systems that are capable of 1) Continuous Learning, 2) Transfer and Adaptation, and 3) Scalability. Unfortunately, efforts to improve these capabilities are typically treated as distinct areas of research that are assessed independently, without regard to the impact of each separate capability on other aspects of the system. We instead propose a holistic approach, using a suite of metrics and an evaluation framework to assess Lifelong Learning in a principled way that is agnostic to specific domains or system techniques. Through five case studies, we show that this suite of metrics can inform the development of varied and complex Lifelong Learning systems. We highlight how the proposed suite of metrics quantifies performance trade-offs present during Lifelong Learning system development - both the widely discussed Stability-Plasticity dilemma and the newly proposed relationship between Sample Efficient and Robust Learning. Further, we make recommendations for the formulation and use of metrics to guide the continuing development of Lifelong Learning systems and assess their progress in the future.
Tighter Regret Analysis and Optimization of Online Federated Learning
Kwon, Dohyeok, Park, Jonghwan, Hong, Songnam
In federated learning (FL), it is commonly assumed that all data are placed at clients in the beginning of machine learning (ML) optimization (i.e., offline learning). However, in many real-world applications, it is expected to proceed in an online fashion. To this end, online FL (OFL) has been introduced, which aims at learning a sequence of global models from decentralized streaming data such that the so-called cumulative regret is minimized. Combining online gradient descent and model averaging, in this framework, FedOGD is constructed as the counterpart of FedSGD in FL. While it can enjoy an optimal sublinear regret, FedOGD suffers from heavy communication costs. In this paper, we present a communication-efficient method (named OFedIQ) by means of intermittent transmission (enabled by client subsampling and periodic transmission) and quantization. For the first time, we derive the regret bound that captures the impact of data-heterogeneity and the communication-efficient techniques. Through this, we efficiently optimize the parameters of OFedIQ such as sampling rate, transmission period, and quantization levels. Also, it is proved that the optimized OFedIQ can asymptotically achieve the performance of FedOGD while reducing the communication costs by 99%. Via experiments with real datasets, we demonstrate the effectiveness of the optimized OFedIQ.
A step-by-step guide to using MLFlow Recipes to refactor messy notebooks
Code repository for this post is here: you can see the MLFlow Recipes template in the main branch and the filled-in template on the fill-in-steps branch. The announcement of MLFlow 2.0 included a new framework called MLFlow Recipes. For a Data Scientists, using MLFlow Recipes means cloning a git repository, or "template", that comes with a ready-to-go folder structure for any regression or binary classification problem. This folder structure includes everything, from library requirements, configuration, notebooks and tests, that's needed to make a data science project reproducible and production-ready. It's easy to start a new project with MLFlow Recipes -- git clone a template from the MLFlow repository, and you are good to go.
AI4AJ 2023
The intended audience for the workshop includes practitioners, researchers, and developers working to employ technology to improve access to justice. The workshop is intended be accessible to attorneys, computer scientists, legal aid workers, and social scientists. The workshop will address technological innovations intended improve access to justice and delivery of services and benefits to citizens and reduction of risks created by such technology, including the due-process, bias, and privacy concerns that can arise from automated support of self-represented litigants. It will also examine human-computer interface issues concerning how and when self-represented litigants (SRLs) should use AI systems.
Progress in the field of Spoken language understanding part1
Abstract: Multilingual spoken language understanding (SLU) consists of two sub-tasks, namely intent detection and slot filling. To improve the performance of these two sub-tasks, we propose to use consistency regularization based on a hybrid data augmentation strategy. The consistency regularization enforces the predicted distributions for an example and its semantically equivalent augmentation to be consistent. We conduct experiments on the MASSIVE dataset under both full-dataset and zero-shot settings. Experimental results demonstrate that our proposed method improves the performance on both intent detection and slot filling tasks.