We consider gradient descent on functions of the form $L_1 = |f|$ and $L_2 = f^2$, where $f: \mathbb{R}^n \rightarrow \mathbb{R}$ is any smooth function with 0 as a regular value. We show that gradient descent implemented with a discrete step size $\tau$ behaves qualitatively differently from continuous gradient descent. We show that over long time scales, continuous and discrete gradient descent on $L_1$ find different minima of $L_1$, and we can characterize the difference - the minima that tend to be found by discrete gradient descent lie in a secondary critical submanifold $M' \subset M$, the locus within $M$ where the function $K=|\nabla f|^2 \big|_M$ is minimized. In this paper, we explain this behavior. We also study the more subtle behavior of discrete gradient descent on $L_2$.

Google has released a beta version of BigQuery ML, new software that lets users build some machine learning models... You forgot to provide an Email Address. This email address doesn't appear to be valid. This email address is already registered. You have exceeded the maximum character limit.

"What gets measured, gets managed " – Peter Drucker The most important aim of all the clustering techniques is to group together the similar data points. K-means clustering algorithm is an unsupervised machine learning algorithm. It is a method of vector quantization that aims at grouping the similar data by minimizing the squared error function. You can apply k-means to any clustering problem provided you are having proper feature vector (Vector-Space Model) from data points and a similarity/distance measure that can measure similarity/distance between the feature vectors. The k-means clustering algorithm is used when you have unlabeled data (i.e., data without defined categories or groups).

A few months ago we released ML.NET 0.1 at //Build 2018., ML.NET is a cross-platform, open source machine learning framework for .NET developers. We've gotten great feedback so far and would like to thank the community for your engagement as we continue to develop ML.NET together in the open. We are happy to announce the latest version: ML.NET 0.4. In this release we've improved support for natural language processing (NLP) scenarios by adding the Word Embedding Transform, improved the speed of linear learners like binary classification and linear regression by adding support for the SymSGD learner, made improvements to the F# API and samples for ML.NET, bug fixes and more. Additionally, we really want your feedback on making ML.NET really easy to use.

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Ensembling word embeddings to improve distributed word representations has shown good success for natural language processing tasks in recent years. These approaches either carry out straightforward mathematical operations over a set of vectors or use unsupervised learning to find a lower-dimensional representation. This work compares meta-embeddings trained for different losses, namely loss functions that account for angular distance between the reconstructed embedding and the target and those that account normalized distances based on the vector length. We argue that meta-embeddings are better to treat the ensemble set equally in unsupervised learning as the respective quality of each embedding is unknown for upstream tasks prior to meta-embedding. We show that normalization methods that account for this such as cosine and KL-divergence objectives outperform meta-embedding trained on standard $\ell_1$ and $\ell_2$ loss on \textit{defacto} word similarity and relatedness datasets and find it outperforms existing meta-learning strategies.

Background: It is still an open research area to theoretically understand why Deep Neural Networks (DNNs)-- equipped with many more parameters than training data and trained by (stochastic) gradient-based methods-- often achieve remarkably low generalization error. Contribution: We study DNN training by Fourier analysis. Our theoretical framework explains: i) DNN with (stochastic) gradient-based methods endows low-frequency components of the target function with a higher priority during the training; ii) Small initialization leads to good generalization ability of DNN while preserving the DNN's ability of fitting any function. These results are further confirmed by experiments of DNNs fitting the following datasets, i.e., natural images, one-dimensional functions and MNIST dataset.

Learning can be seen as approximating an unknown function by interpolating the training data. Kriging offers a solution to this problem based on the prior specification of a kernel. We explore a numerical approximation approach to kernel selection/construction based on the simple premise that a kernel must be good if the number of interpolation points can be halved without significant loss in accuracy (measured using the intrinsic RKHS norm $\|\cdot\|$ associated with the kernel). We first test and motivate this idea on a simple problem of recovering the Green's function of an elliptic PDE (with inhomogeneous coefficients) from the sparse observation of one of its solutions. Next we consider the problem of learning non-parametric families of deep kernels of the form $K_1(F_n(x),F_n(x'))$ with $F_{n+1}=(I_d+\epsilon G_{n+1})\circ F_n$ and $G_{n+1} \in \operatorname{Span}\{K_1(F_n(x_i),\cdot)\}$. With the proposed approach constructing the kernel becomes equivalent to integrating a stochastic data driven dynamical system, which allows for the training of very deep (bottomless) networks and the exploration of their properties. These networks learn by constructing flow maps in the kernel and input spaces via incremental data-dependent deformations/perturbations (appearing as the cooperative counterpart of adversarial examples) and, at profound depths, they (1) can achieve accurate classification from only one data point per class (2) appear to learn archetypes of each class (3) expand distances between points that are in different classes and contract distances between points in the same class. For kernels parameterized by the weights of Convolutional Neural Network, minimizing approximation errors incurred by halving random subsets of interpolation points, appears to outperform training (the same CNN architecture) with relative entropy and dropout.

Acute Kidney Injury (AKI), a sudden decline in kidney function, is associated with increased mortality, morbidity, length of stay, and hospital cost. Since AKI is sometimes preventable, there is great interest in prediction. Most existing studies consider all patients and therefore restrict to features available in the first hours of hospitalization. Here, the focus is instead on rehospitalized patients, a cohort in which rich longitudinal features from prior hospitalizations can be analyzed. Our objective is to provide a risk score directly at hospital re-entry. Gradient boosting, penalized logistic regression (with and without stability selection), and a recurrent neural network are trained on two years of adult inpatient EHR data (3,387 attributes for 34,505 patients who generated 90,013 training samples with 5,618 cases and 84,395 controls). Predictions are internally evaluated with 50 iterations of 5-fold grouped cross-validation with special emphasis on calibration, an analysis of which is performed at the patient as well as hospitalization level. Error is assessed with respect to diagnosis, race, age, gender, AKI identification method, and hospital utilization. In an additional experiment, the regularization penalty is severely increased to induce parsimony and interpretability. Predictors identified for rehospitalized patients are also reported with a special analysis of medications that might be modifiable risk factors. Insights from this study might be used to construct a predictive tool for AKI in rehospitalized patients. An accurate estimate of AKI risk at hospital entry might serve as a prior for an admitting provider or another predictive algorithm.

Yinlam Chow DeepMind We study risk-sensitive imitation learning where the agent's goal is to perform at least as well as the expert in terms of a risk profile. We first formulate our risk-sensitive imitation learning setting. We consider the generative adversarial approach to imitation learning (GAIL) and derive an optimization problem for our formulation, which we call risk-sensitive GAIL (RS-GAIL). We then derive two different versions of our RS-GAIL optimization problem that aim at matching the risk profiles of the agent and the expert w.r.t. Jensen-Shannon (JS) divergence and Wasserstein distance, and develop risk-sensitive generative adversarial imitation learning algorithms based on these optimization problems. We evaluate the performance of our JSbased algorithm and compare it with GAIL and the risk-averse imitation learning (RAIL) algorithm in two Mu-JoCo tasks.