Summary: I learn best with toy code that I can play with. This tutorial teaches gradient descent via a very simple toy example, a short python implementation. Followup Post: I intend to write a followup post to this one adding popular features leveraged by state-of-the-art approaches (likely Dropout, DropConnect, and Momentum). Feel free to follow if you'd be interested in reading more and thanks for all the feedback! In Part 1, I laid out the basis for backpropagation in a simple neural network. Backpropagation allowed us to measure how each weight in the network contributed to the overall error. This ultimately allowed us to change these weights using a different algorithm, Gradient Descent.

I had thought my job was safe from automation--a computer couldn't possibly replicate the complex creativity of human language in writing or piece together a coherent story. I may have been wrong. Authors beware, because an AI-written novel just made it past the first round of screening for a national literary prize in Japan. The novel this program co-authored is titled, The Day A Computer Writes A Novel. It was entered into a writing contest for the Hoshi Shinichi Literary Award.

Deloitte today announced an alliance with Kira Systems to bring the power of machine learning to the workplace, an innovation that could help free workers from the tedium of reviewing contracts and other documents. The alliance will combine Deloitte's business insights in cognitive technologies with Kira Systems' advances in machine-learning in creating models that quickly "read" thousands of complex documents, extracting and structuring textual information for better analysis. This capability holds broad applications for the marketplace, said Craig Muraskin, Deloitte LLP, managing director of Deloitte's US Innovation group, as the extensive review of documents goes into many pressing business activities, including investigations, mergers, contract management, and leasing arrangements. "Wading through miles of corporate jargon hunting for key words and patterns can consume considerable time and resources," said Muraskin. "By teaming with Kira Systems we can help organizations reduce their review time while redeploying talent to higher value activities--let's save our eyes for more strategic matters." Noah Waisberg, CEO of Kira Systems, said recent innovations by his company, such as Kira Quick Study, are graduating machine learning to new levels of accomplishment.

Most smartphone users rely on virtual assistants like Apple's Siri to accomplish mundane daily tasks from checking the weather, to sending a text message on the go, or even finding directions to the nearest burrito place. But what happens in times of crisis or distress? A recent study commissioned by Stanford University and the University of California concluded that several smartphone AIs from Apple, Samsung, Google and Microsoft might not be so helpful after all. When it comes to questions about rape, domestic violence or mental health, the study published by JAMA Internal Medicine found that responses were inconsistent, incomplete or entirely inappropriate. Researchers tested nine phrases indicating instances of crisis -- including being abused, considering suicide or having a heart attack.

Top Machine Learning & Data Mining Books - for this post, we have scraped various signals (e.g. We have combined all signals to compute a Quality Score for each book and publish the list of top Machine Learning and Data Mining books. The readers will love the list because it is data-driven & objective. This book is very well rated on Amazon website and is written by three professors from USC, Stanford and University of Washington. The three authors: Gareth James, Daniela Witten, & Trevor Hastie all have backgrounds in statistics.

Collaborative filtering is the process of filtering for information using techniques involving collaboration among multiple agents. Applications of collaborative filtering typically involve very large data sets. This article covers some good tutorials regarding collaborative filtering we came across in Python, Java and R. Crab engine aims to provide a rich set of components from which you can construct a customized recommender system from a set of algorithms. The tutorial is from official documentation of Crab. This article presents an implementation of the collaborative filtering algorithm, that filters information for a user based on a collection of user profiles.

The problem of recovering a structured signal $\mathbf{x} \in \mathbb{C}^p$ from a set of dimensionality-reduced linear measurements $\mathbf{b} = \mathbf {A}\mathbf {x}$ arises in a variety of applications, such as medical imaging, spectroscopy, Fourier optics, and computerized tomography. Due to computational and storage complexity or physical constraints imposed by the problem, the measurement matrix $\mathbf{A} \in \mathbb{C}^{n \times p}$ is often of the form $\mathbf{A} = \mathbf{P}_{\Omega}\boldsymbol{\Psi}$ for some orthonormal basis matrix $\boldsymbol{\Psi}\in \mathbb{C}^{p \times p}$ and subsampling operator $\mathbf{P}_{\Omega}: \mathbb{C}^{p} \rightarrow \mathbb{C}^{n}$ that selects the rows indexed by $\Omega$. This raises the fundamental question of how best to choose the index set $\Omega$ in order to optimize the recovery performance. Previous approaches to addressing this question rely on non-uniform \emph{random} subsampling using application-specific knowledge of the structure of $\mathbf{x}$. In this paper, we instead take a principled learning-based approach in which a \emph{fixed} index set is chosen based on a set of training signals $\mathbf{x}_1,\dotsc,\mathbf{x}_m$. We formulate combinatorial optimization problems seeking to maximize the energy captured in these signals in an average-case or worst-case sense, and we show that these can be efficiently solved either exactly or approximately via the identification of modularity and submodularity structures. We provide both deterministic and statistical theoretical guarantees showing how the resulting measurement matrices perform on signals differing from the training signals, and we provide numerical examples showing our approach to be effective on a variety of data sets.

This paper tackles the problem of endogenous link prediction for knowledge base completion. Knowledge bases can be represented as directed graphs whose nodes correspond to entities and edges to relationships. Previous attempts either consist of powerful systems with high capacity to model complex connectivity patterns, which unfortunately usually end up overfitting on rare relationships, or in approaches that trade capacity for simplicity in order to fairly model all relationships, frequent or not. In this paper, we propose Tatec, a happy medium obtained by complementing a high-capacity model with a simpler one, both pre-trained separately and then combined. We present several variants of this model with different kinds of regularization and combination strategies and show that this approach outperforms existing methods on different types of relationships by achieving state-of-the-art results on four benchmarks of the literature.

We represent knowledge as integrity constraints in a formalization of probabilistic spatio-temporal knowledge bases. We start by defining the syntax and semantics of a formalization called PST knowledge bases. This definition generalizes an earlier version, called SPOT, which is a declarative framework for the representation and processing of probabilistic spatio-temporal data where probability is represented as an interval because the exact value is unknown. We augment the previous definition by adding a type of non-atomic formula that expresses integrity constraints. The result is a highly expressive formalism for knowledge representation dealing with probabilistic spatio-temporal data. We obtain complexity results both for checking the consistency of PST knowledge bases and for answering queries in PST knowledge bases, and also specify tractable cases. All the domains in the PST framework are finite, but we extend our results also to arbitrarily large finite domains.

This work provides improved guarantees for streaming principle component analysis (PCA). Given $A_1, \ldots, A_n\in \mathbb{R}^{d\times d}$ sampled independently from distributions satisfying $\mathbb{E}[A_i] = \Sigma$ for $\Sigma \succeq \mathbf{0}$, this work provides an $O(d)$-space linear-time single-pass streaming algorithm for estimating the top eigenvector of $\Sigma$. The algorithm nearly matches (and in certain cases improves upon) the accuracy obtained by the standard batch method that computes top eigenvector of the empirical covariance $\frac{1}{n} \sum_{i \in [n]} A_i$ as analyzed by the matrix Bernstein inequality. Moreover, to achieve constant accuracy, our algorithm improves upon the best previous known sample complexities of streaming algorithms by either a multiplicative factor of $O(d)$ or $1/\mathrm{gap}$ where $\mathrm{gap}$ is the relative distance between the top two eigenvalues of $\Sigma$. These results are achieved through a novel analysis of the classic Oja's algorithm, one of the oldest and most popular algorithms for streaming PCA. In particular, this work shows that simply picking a random initial point $w_0$ and applying the update rule $w_{i + 1} = w_i + \eta_i A_i w_i$ suffices to accurately estimate the top eigenvector, with a suitable choice of $\eta_i$. We believe our result sheds light on how to efficiently perform streaming PCA both in theory and in practice and we hope that our analysis may serve as the basis for analyzing many variants and extensions of streaming PCA.