Learning has been essential to our existence. And, continuous learning helps an individual avoid stagnation in any profession and ensures that one keeps on moving forward towards reaching his goal and potential. The same also goes for machine models that are backed by AI's machine learning algorithm. Continual learning, also called lifelong learning or online machine learning, is a fundamental idea in machine learning in which models continuously learn and evolve based on the input of increasing amounts of data while retaining previously learned knowledge. In practice, this refers to supporting a model's ability to autonomously learn and adapt in production as new data comes in.

This article is part of "AI education", a series of posts that review and explore educational content on data science and machine learning. Teaching yourself Python machine learning can be a daunting task if you don't know where to start. Fortunately, there are plenty of good introductory books and online courses that teach you the basics. It is the advanced books, however, that teach you the skills you need to decide which algorithm better solves a problem and which direction to take when tuning hyperparameters. A while ago, I was introduced to Machine Learning Algorithms, Second Edition by Giuseppe Bonaccorso, a book that almost falls into the latter category. While the title sounds like another introductory book on machine learning algorithms, the content is anything but.

Deep Reinforcement Learning Course is a free course (articles and videos) about Deep Reinforcement Learning, where we'll learn the main algorithms, and how to implement them in Tensorflow and PyTorch.

When Haley, a sophomore at Indiana University, took a test for an accounting class in September, she--like many college students during this pandemic--was sitting not in a classroom but in her bedroom. And instead of a teacher watching for signs of cheating, there was something new: an AI, studying Haley's every move through her laptop's webcam. The university was conducting remote exams using Respondus, a type of "online proctoring" software. The software locks down a student's desktop so they can't switch tabs to Google an answer, and then it uses visual AI to examine--among other things--their head movements to judge whether they're looking somewhere other than at the screen. Haley's head was setting off alarms.

Teaching yourself Python machine learning can be a daunting task if you don't know where to start. Fortunately, there are plenty of good introductory books and online courses that teach you the basics. It is the advanced books, however, that teach you the skills you need to decide which algorithm better solves a problem and which direction to take when tuning hyperparameters. A while ago, I was introduced to Machine Learning Algorithms, Second Edition by Giuseppe Bonaccorso, a book that almost falls into the latter category. While the title sounds like another introductory book on machine learning algorithms, the content is anything but.

It's been nearly 4 years since Tensorflow was released, and the library has evolved to its official second version. Tensorflow is Google's library for deep learning and artificial intelligence. Tensorflow is the world's most popular library for deep learning, and it's built by Google, whose parent Alphabet recently became the most cash-rich company in the world (just a few days before I wrote this). It is the library of choice for many companies doing AI and machine learning. In other words, if you want to do deep learning, you gotta know Tensorflow.

We apply two independent data analysis methodologies to locate stable climate states in an intermediate complexity climate model. First, drawing from the theory of quasipotentials, and viewing the state space as an energy landscape with valleys and mountain ridges, we infer the relative likelihood of the identified multistable climate states, and investigate the most likely transition trajectories as well as the expected transition times between them. Second, harnessing techniques from data science, specifically manifold learning, we characterize the data landscape of the simulation data to find climate states and basin boundaries within a fully agnostic and unsupervised framework. Both approaches show remarkable agreement, and reveal, apart from the well known warm and snowball earth states, a third intermediate stable state in one of the two climate models we consider. The combination of our approaches allows to identify how the negative feedback of ocean heat transport and entropy production via the hydrological cycle drastically change the topography of the dynamical landscape of Earth's climate.

We study the effectiveness of replacing the split strategy for the state-of-the-art online tree learner, Hoeffding Tree, with a rigorous but more eager splitting strategy. Our method, Hoeffding AnyTime Tree (HATT), uses the Hoeffding Test to determine whether the current best candidate split is superior to the current split, with the possibility of revision, while Hoeffding Tree aims to determine whether the top candidate is better than the second best and fixes it for all posterity. Our method converges to the ideal batch tree while Hoeffding Tree does not. Decision tree ensembles are widely used in practice, and in this work, we study the efficacy of HATT as a base learner for online bagging and online boosting ensembles. On UCI and synthetic streams, the success of Hoeffding AnyTime Tree in terms of prequential accuracy over Hoeffding Tree is established. HATT as a base learner component outperforms HT within a 0.05 significance level for the majority of tested ensembles on what we believe is the largest and most comprehensive set of testbenches in the online learning literature. Our results indicate that HATT is a superior alternative to Hoeffding Tree in a large number of ensemble settings.

In display advertising, a small group of sellers and bidders face each other in up to 10 12 auctions a day. In this context, revenue maximisation via monopoly price learning is a high-value problem for sellers. By nature, these auctions are online and produce a very high frequency stream of data. This results in a computational strain that requires algorithms be real-time. Unfortunately, existing methods inherited from the batch setting suffer O($\sqrt t$) time/memory complexity at each update, prohibiting their use. In this paper, we provide the first algorithm for online learning of monopoly prices in online auctions whose update is constant in time and memory.

Structural equation models (SEMs) are widely used in sciences, ranging from economics to psychology, to uncover causal relationships underlying a complex system under consideration and estimate structural parameters of interest. We study estimation in a class of generalized SEMs where the object of interest is defined as the solution to a linear operator equation. We formulate the linear operator equation as a min-max game, where both players are parameterized by neural networks (NNs), and learn the parameters of these neural networks using the stochastic gradient descent. We consider both 2-layer and multi-layer NNs with ReLU activation functions and prove global convergence in an overparametrized regime, where the number of neurons is diverging. The results are established using techniques from online learning and local linearization of NNs, and improve in several aspects the current state-of-the-art. For the first time we provide a tractable estimation procedure for SEMs based on NNs with provable convergence and without the need for sample splitting.