Instructional Theory

Isotonic Regression is THE Coolest Machine-Learning Model You Might Not Have Heard Of


Isotonic regression is a free-form linear model that can be fit to predict sequences of observations. However, there are two major differences between isotonic regression and a similar model like weighted least squares. An isotonic function must not be non-decreasing. This is because an isotonic function is a monotonic function, meaning a function that preserves or reverses a given order. Isotonic regressors use an interesting concept called " Order Theory."

Why you should care about debugging machine learning models


Receive weekly insight from industry insiders--plus exclusive content, offers, and more on the topic of AI.

Arthur announces $3.3M seed to monitor machine learning model performance – TechCrunch


Machine learning is a complex process. You build a model, test it in laboratory conditions, then put it out in the world. After that, how do you monitor how well it's tracking what you designed it to do? Arthur wants to help, and today it emerged from stealth with a new platform to help you monitor machine learning models in production. The company also announced it had closed a $3.3 million seed round, which closed in August.

Major U.S. bank, a pioneer in the use of machine learning models, teams with Protiviti to improve its model validation framework


Following the financial crisis of 2007-2008, regulators issued specific guidance to help banks reduce the risk of financial losses or other adverse consequences stemming from decisions based on incorrect or misused financial models. Since then, the guidance has become the model risk management bible for financial institutions. It is used to ensure that model validation, typically performed annually, can identify vulnerabilities in the models and manage them effectively. Recently, the rapid advance and broader adoption of machine learning (ML) models have added more complexity and time to the model validation process. Specifically, ML models have highlighted expertise gaps in in-house model validation teams trained in traditional modeling techniques.

Humanoid robot rolls 360 after 25 minutes, using reinforcement learning model


Sign in to report inappropriate content. Robot being trained for 500 iterations to learn to control inclination of torso. This is done by selecting goal inclinations the robot attempts to reach, while creating a network of postures to move between. In the final evaluation (0:49), goals are selected manually, to force the robot to roll around 360 degrees to reach them.

How to build a deep learning model in 15 minutes


As Instacart has grown, we've learned a few things the hard way. We're open sourcing Lore, a framework to make machine learning approachable for Engineers and maintainable for Machine Learning Researchers.

We'll Do It Live: Updating Machine Learning Models on Flask/uWSGI with No Downtime


Flask is one of the most popular REST API frameworks used for hosting machine learning (ML) models. The choice is heavily influenced by a data science team's expertise in Python and the reusability of training assets built in Python. At WW, Flask is used extensively by the data science team to serve predictions from various ML models. However, there are a few considerations that need to be made before a Flask application is production-ready. If Flask code isn't modified to run asynchronously, it only can run one request per process at a time.

Amazon SageMaker Model Monitor – Fully Managed Automatic Monitoring For Your Machine Learning Models Amazon Web Services


Today, we're extremely happy to announce Amazon SageMaker Model Monitor, a new capability of Amazon SageMaker that automatically monitors machine learning (ML) models in production, and alerts you when data quality issues appear. The first thing I learned when I started working with data is that there is no such thing as paying too much attention to data quality. Raise your hand if you've spent hours hunting down problems caused by unexpected NULL values or by exotic character encodings that somehow ended up in one of your databases. As models are literally built from large amounts of data, it's easy to see why ML practitioners spend so much time caring for their data sets. In particular, they make sure that data samples in the training set (used to train the model) and in the validation set (used to measure its accuracy) have the same statistical properties.

Leader Stochastic Gradient Descent for Distributed Training of Deep Learning Models

Neural Information Processing Systems

We consider distributed optimization under communication constraints for training deep learning models. We propose a new algorithm, whose parameter updates rely on two forces: a regular gradient step, and a corrective direction dictated by the currently best-performing worker (leader). Our method differs from the parameter-averaging scheme EASGD in a number of ways: (i) our objective formulation does not change the location of stationary points compared to the original optimization problem; (ii) we avoid convergence decelerations caused by pulling local workers descending to different local minima to each other (i.e. to the average of their parameters); (iii) our update by design breaks the curse of symmetry (the phenomenon of being trapped in poorly generalizing sub-optimal solutions in symmetric non-convex landscapes); and (iv) our approach is more communication efficient since it broadcasts only parameters of the leader rather than all workers. We provide theoretical analysis of the batch version of the proposed algorithm, which we call Leader Gradient Descent (LGD), and its stochastic variant (LSGD). Finally, we implement an asynchronous version of our algorithm and extend it to the multi-leader setting, where we form groups of workers, each represented by its own local leader (the best performer in a group), and update each worker with a corrective direction comprised of two attractive forces: one to the local, and one to the global leader (the best performer among all workers).