Interest in learning machine learning has skyrocketed in the years since Harvard Business Review article named'Data Scientist' the'Sexiest job of the 21st century'. But if you're just starting out in machine learning, it can be a bit difficult to break into. It has been reposted with permission, and was last updated in 2019). This post is targeted towards beginners. If you've got some experience in data science and machine learning, you may be more interested in this more in-depth tutorial on doing machine learning in Python with scikit-learn, or in our machine learning courses, which start here. If you're not clear yet on the differences between "data science" and "machine learning," this article offers a good explanation: machine learning and data science -- what makes them different? Machine learning algorithms are programs that can learn from data and improve from experience, without human intervention.

Thus a new instance and a previously seen example might be similar in an abstract (non-explicit) sense, but that similarity is still incorporated into the kernel. When Einstein invented Special Relativity he was not exactly aping another physical theory he had seen before, but at an abstract level the physical constraint (speed of light constant in all reference frames) and algebraic incorporation of this fact into a description of spacetime (Lorentz symmetry) may have been "similar" to examples he had seen already in simple geometry / algebra.

This article is part 2 of my Popular Machine Learning Interview questions. Here I feature more questions I usually see asked during interviews. I shall note that this isn't an interview prep guide nor a conclusive list of all questions. Rather, you should use this article as a refresher for your Machine Learning knowledge. I suggest reading the question then try to answer it yourself before reading the answer.

Unsupervised domain adaptation aims to learn a task classifier that performs well on the unlabeled target domain, by utilizing the labeled source domain. Inspiring results have been acquired by learning domain-invariant deep features via domain-adversarial training. However, its parallel design of task and domain classifiers limits the ability to achieve a finer category-level domain alignment. To promote categorical domain adaptation (CatDA), based on a joint category-domain classifier, we propose novel losses of adversarial training at both domain and category levels. Since the joint classifier can be regarded as a concatenation of individual task classifiers respectively for the two domains, our design principle is to enforce consistency of category predictions between the two task classifiers. Moreover, we propose a concept of vicinal domains whose instances are produced by a convex combination of pairs of instances respectively from the two domains. Intuitively, alignment of the possibly infinite number of vicinal domains enhances that of original domains. We propose novel adversarial losses for vicinal domain adaptation (VicDA) based on CatDA, leading to Vicinal and Categorical Domain Adaptation (ViCatDA). We also propose Target Discriminative Structure Recovery (TDSR) to recover the intrinsic target discrimination damaged by adversarial feature alignment. We also analyze the principles underlying the ability of our key designs to align the joint distributions. Extensive experiments on several benchmark datasets demonstrate that we achieve the new state of the art.

Datasets play a critical role in shaping the perception of performance and progress in machine learning (ML)--the way we collect, process, and analyze data affects the way we benchmark success and form new research agendas (Paullada et al., 2020; Dotan & Milli, 2020). A growing appreciation of this determinative role of datasets has sparked a concomitant concern that standard datasets used for training and evaluating ML models lack diversity along significant dimensions, for example, geography, gender, and skin type (Shankar et al., 2017; Buolamwini & Gebru, 2018). Lack of diversity in evaluation data can obfuscate disparate performance when evaluating based on aggregate accuracy (Buolamwini & Gebru, 2018). Lack of diversity in training data can limit the extent to which learned models can adequately apply to all portions of a population, a concern highlighted in recent work in the medical domain (Habib et al., 2019; Hofmanninger et al., 2020). Our work aims to develop a general unifying perspective on the way that dataset composition affects outcomes of machine learning systems.

Bayesian optimization has emerged as a powerful strategy to accelerate scientific discovery by means of autonomous experimentation. However, expensive measurements are required to accurately estimate materials properties, and can quickly become a hindrance to exhaustive materials discovery campaigns. Here, we introduce Gemini: a data-driven model capable of using inexpensive measurements as proxies for expensive measurements by correcting systematic biases between property evaluation methods. We recommend using Gemini for regression tasks with sparse data and in an autonomous workflow setting where its predictions of expensive to evaluate objectives can be used to construct a more informative acquisition function, thus reducing the number of expensive evaluations an optimizer needs to achieve desired target values. In a regression setting, we showcase the ability of our method to make accurate predictions of DFT calculated bandgaps of hybrid organic-inorganic perovskite materials. We further demonstrate the benefits that Gemini provides to autonomous workflows by augmenting the Bayesian optimizer Phoenics to yeild a scalable optimization framework leveraging multiple sources of measurement. Finally, we simulate an autonomous materials discovery platform for optimizing the activity of electrocatalysts for the oxygen evolution reaction. Realizing autonomous workflows with Gemini, we show that the number of measurements of a composition space comprising expensive and rare metals needed to achieve a target overpotential is significantly reduced when measurements from a proxy composition system with less expensive metals are available.

This paper presents a machine learning framework (GP-NODE) for Bayesian systems identification from partial, noisy and irregular observations of nonlinear dynamical systems. The proposed method takes advantage of recent developments in differentiable programming to propagate gradient information through ordinary differential equation solvers and perform Bayesian inference with respect to unknown model parameters using Hamiltonian Monte Carlo sampling and Gaussian Process priors over the observed system states. This allows us to exploit temporal correlations in the observed data, and efficiently infer posterior distributions over plausible models with quantified uncertainty. Moreover, the use of sparsity-promoting priors such as the Finnish Horseshoe for free model parameters enables the discovery of interpretable and parsimonious representations for the underlying latent dynamics. A series of numerical studies is presented to demonstrate the effectiveness of the proposed GP-NODE method including predator-prey systems, systems biology, and a 50-dimensional human motion dynamical system. Taken together, our findings put forth a novel, flexible and robust workflow for data-driven model discovery under uncertainty. All code and data accompanying this manuscript are available online at \url{https://github.com/PredictiveIntelligenceLab/GP-NODEs}.

Gaussian processes (GPs) are frequently used in machine learning and statistics to construct powerful models. In a Bayesian setting, GPs provide a probabilistic approach to model unknown functions; specifically, the GP prior assumes that the function evaluated at any finite set of inputs has a Gaussian distribution with consistent parameters, specified by the mean function and symmetric positive definite covariance (or kernel) function. The flexible, probabilistic and nonparametric nature of GP models makes them appropriate and useful in a wide range of applications, including geostatistics [Matheron, 1973], atmospheric sciences [Berrocal et al., 2010], biology [Stathopoulos et al., 2014], inverse problems [Kaipio and Somersalo, 2006], and more. However, when employing GPs in practice, important considerations must be made, specifically, to address the computational burden, approximation of the posterior, form of the covariance function and inference of its hyperparmeters. First, GPs suffer from a high computational burden, due to the need to store and invert large and dense covariance matrices.

Ensemble-based modifications of the well-known SHapley Additive exPlanations (SHAP) method for the local explanation of a black-box model are proposed. The modifications aim to simplify SHAP which is computationally expensive when there is a large number of features. The main idea behind the proposed modifications is to approximate SHAP by an ensemble of SHAPs with a smaller number of features. According to the first modification, called ER-SHAP, several features are randomly selected many times from the feature set, and Shapley values for the features are computed by means of "small" SHAPs. The explanation results are averaged to get the final Shapley values. According to the second modification, called ERW-SHAP, several points are generated around the explained instance for diversity purposes, and results of their explanation are combined with weights depending on distances between points and the explained instance. The third modification, called ER-SHAP-RF, uses the random forest for preliminary explanation of instances and determining a feature probability distribution which is applied to selection of features in the ensemble-based procedure of ER-SHAP. Many numerical experiments illustrating the proposed modifications demonstrate their efficiency and properties for local explanation.

We provide a unifying view of a large family of previous imitation learning algorithms through the lens of moment matching. At its core, our classification scheme is based on whether the learner attempts to match (1) reward or (2) action-value moments of the expert's behavior, with each option leading to differing algorithmic approaches. By considering adversarially chosen divergences between learner and expert behavior, we are able to derive bounds on policy performance that apply for all algorithms in each of these classes, the first to our knowledge. We also introduce the notion of recoverability, implicit in many previous analyses of imitation learning, which allows us to cleanly delineate how well each algorithmic family is able to mitigate compounding errors. We derive two novel algorithm templates, AdVIL and AdRIL, with strong guarantees, simple implementation, and competitive empirical performance.