Instructional Material
Machine Learning for All
Machine Learning, often called Artificial Intelligence or AI, is one of the most exciting areas of technology at the moment. We see daily news stories that herald new breakthroughs in facial recognition technology, self driving cars or computers that can have a conversation just like a real person. Machine Learning technology is set to revolutionise almost any area of human life and work, and so will affect all our lives, and so you are likely to want to find out more about it. Machine Learning has a reputation for being one of the most complex areas of computer science, requiring advanced mathematics and engineering skills to understand it. While it is true that working as a Machine Learning engineer does involve a lot of mathematics and programming, we believe that anyone can understand the basic concepts of Machine Learning, and given the importance of this technology, everyone should.
A Comparison of SVM against Pre-trained Language Models (PLMs) for Text Classification Tasks
Wahba, Yasmen, Madhavji, Nazim, Steinbacher, John
The emergence of pre-trained language models (PLMs) has shown great success in many Natural Language Processing (NLP) tasks including text classification. Due to the minimal to no feature engineering required when using these models, PLMs are becoming the de facto choice for any NLP task. However, for domain-specific corpora (e.g., financial, legal, and industrial), fine-tuning a pre-trained model for a specific task has shown to provide a performance improvement. In this paper, we compare the performance of four different PLMs on three public domain-free datasets and a real-world dataset containing domain-specific words, against a simple SVM linear classifier with TFIDF vectorized text. The experimental results on the four datasets show that using PLMs, even fine-tuned, do not provide significant gain over the linear SVM classifier. Hence, we recommend that for text classification tasks, traditional SVM along with careful feature engineering can pro-vide a cheaper and superior performance than PLMs.
Deep Surrogate Docking: Accelerating Automated Drug Discovery with Graph Neural Networks
Hosseini, Ryien, Simini, Filippo, Clyde, Austin, Ramanathan, Arvind
The process of screening molecules for desirable properties is a key step in several applications, ranging from drug discovery to material design. During the process of drug discovery specifically, protein-ligand docking, or chemical docking, is a standard in-silico scoring technique that estimates the binding affinity of molecules with a specific protein target. Recently, however, as the number of virtual molecules available to test has rapidly grown, these classical docking algorithms have created a significant computational bottleneck. We address this problem by introducing Deep Surrogate Docking (DSD), a framework that applies deep learning-based surrogate modeling to accelerate the docking process substantially. DSD can be interpreted as a formalism of several earlier surrogate prefiltering techniques, adding novel metrics and practical training practices. Specifically, we show that graph neural networks (GNNs) can serve as fast and accurate estimators of classical docking algorithms. Additionally, we introduce FiLMv2, a novel GNN architecture which we show outperforms existing state-of-the-art GNN architectures, attaining more accurate and stable performance by allowing the model to filter out irrelevant information from data more efficiently. Through extensive experimentation and analysis, we show that the DSD workflow combined with the FiLMv2 architecture provides a 9.496x speedup in molecule screening with a < 3% recall error rate on an example docking task.
Predictive Querying for Autoregressive Neural Sequence Models
Boyd, Alex, Showalter, Sam, Mandt, Stephan, Smyth, Padhraic
In reasoning about sequential events it is natural to pose probabilistic queries such as "when will event A occur next" or "what is the probability of A occurring before B", with applications in areas such as user modeling, medicine, and finance. However, with machine learning shifting towards neural autoregressive models such as RNNs and transformers, probabilistic querying has been largely restricted to simple cases such as next-event prediction. This is in part due to the fact that future querying involves marginalization over large path spaces, which is not straightforward to do efficiently in such models. In this paper we introduce a general typology for predictive queries in neural autoregressive sequence models and show that such queries can be systematically represented by sets of elementary building blocks. We leverage this typology to develop new query estimation methods based on beam search, importance sampling, and hybrids. Across four large-scale sequence datasets from different application domains, as well as for the GPT-2 language model, we demonstrate the ability to make query answering tractable for arbitrary queries in exponentially-large predictive path-spaces, and find clear differences in cost-accuracy tradeoffs between search and sampling methods.
Exploratory Data Analysis for Machine Learning
This first course in the IBM Machine Learning Professional Certificate introduces you to Machine Learning and the content of the professional certificate. In this course you will realize the importance of good, quality data. You will learn common techniques to retrieve your data, clean it, apply feature engineering, and have it ready for preliminary analysis and hypothesis testing. By the end of this course you should be able to: Retrieve data from multiple data sources: SQL, NoSQL databases, APIs, Cloud Describe and use common feature selection and feature engineering techniques Handle categorical and ordinal features, as well as missing values Use a variety of techniques for detecting and dealing with outliers Articulate why feature scaling is important and use a variety of scaling techniques Who should take this course? This course targets aspiring data scientists interested in acquiring hands-on experience with Machine Learning and Artificial Intelligence in a business setting.
Tutorial 2021 AI Act
HAI NET General Under Vision, add a dedicated task Legal Protection by Design This project aims to take seriously the fact that the development and deployment of AI systems is not above the law, as decided in constitutional democracies. This feeds into the task of addressing the question of incorporation of fundamental rights protection into the architecture of AI systems including (1) checks and balances of the Rule of Law and (2) requirements imposed by positive law that elaborates fundamental rights protection. A key result of this task will be a report on a coherent set of design principles firmly grounded in relevant positive law, with a clear emphasis on European law (both EU and Council of Europe). To help developers understand the core tenets of the EU legal framework, we have developed two tutorials, one in 2020 on Legal Protection by Design in relation to EU data protection law [hyperlink to Tutorial 2020] and one in 2021 on the European Commission’s proposal of an EU AI Act [hyperlink to Tutorial 2021]. In the Fall of 2022 we will follow up with a Tutorial on the proposed EU AI Liability Directive. Our findings will entail: - A sufficiently detailed overview of legally relevant roles, such as end-users, targeted persons, software developers, hardware manufacturers, those who put AI applications on the market, platforms that integrate service provision both vertical and horizontal, providers of infrastructure (telecom providers, cloud providers, providers of cyber-physical infrastructure, smart grid providers, etc.); - A sufficiently detailed legal vocabulary, explained at the level of AI applications, such as legal subjects, legal objects, legal rights and obligations, private law liability, fundamental rights protection; - High level principles that anchor the Rule of Law: transparency (e.g. explainability, preregistration of research design), accountability (e.g. clear attribution of tort liability, fines by relevant supervisors, criminal law liability), contestability (e.g. the repertoire of legal remedies, adversarial structure of legal procedure).
Machine Learning: Clustering & Retrieval
A reader is interested in a specific news article and you want to find similar articles to recommend. What is the right notion of similarity? Moreover, what if there are millions of other documents? Each time you want to a retrieve a new document, do you need to search through all other documents? How do you group similar documents together?