We propose random hinge forests, a simple, efficient, and novel variant of decision forests. Importantly, random hinge forests can be readily incorporated as a general component within arbitrary computation graphs that are optimized end-to-end with stochastic gradient descent or variants thereof. We derive random hinge forest and ferns, focusing on their sparse and efficient nature, their min-max margin property, strategies to initialize them for arbitrary network architectures, and the class of optimizers most suitable for optimizing random hinge forest. The performance and versatility of random hinge forests are demonstrated by experiments incorporating a variety of of small and large UCI machine learning data sets and also ones involving the MNIST, Letter, and USPS image datasets. We compare random hinge forests with random forests and the more recent backpropagating deep neural decision forests.

Graph convolutional networks (GCNs) are powerful deep neural networks for graph-structured data. However, GCN computes the representation of a node recursively from its neighbors, making the receptive field size grow exponentially with the number of layers. Previous attempts on reducing the receptive field size by subsampling neighbors do not have a convergence guarantee, and their receptive field size per node is still in the order of hundreds. In this paper, we develop control variate based algorithms which allow sampling an arbitrarily small neighbor size. Furthermore, we prove new theoretical guarantee for our algorithms to converge to a local optimum of GCN. Empirical results show that our algorithms enjoy a similar convergence with the exact algorithm using only two neighbors per node. The runtime of our algorithms on a large Reddit dataset is only one seventh of previous neighbor sampling algorithms.

Information extraction and user intention identification are central topics in modern query understanding and recommendation systems. In this paper, we propose DeepProbe, a generic information-directed interaction framework which is built around an attention-based sequence to sequence (seq2seq) recurrent neural network. DeepProbe can rephrase, evaluate, and even actively ask questions, leveraging the generative ability and likelihood estimation made possible by seq2seq models. DeepProbe makes decisions based on a derived uncertainty (entropy) measure conditioned on user inputs, possibly with multiple rounds of interactions. Three applications, namely a rewritter, a relevance scorer and a chatbot for ad recommendation, were built around DeepProbe, with the first two serving as precursory building blocks for the third. We first use the seq2seq model in DeepProbe to rewrite a user query into one of standard query form, which is submitted to an ordinary recommendation system. Secondly, we evaluate DeepProbe's seq2seq model-based relevance scoring. Finally, we build a chatbot prototype capable of making active user interactions, which can ask questions that maximize information gain, allowing for a more efficient user intention idenfication process. We evaluate first two applications by 1) comparing with baselines by BLEU and AUC, and 2) human judge evaluation. Both demonstrate significant improvements compared with current state-of-the-art systems, proving their values as useful tools on their own, and at the same time laying a good foundation for the ongoing chatbot application.

Information bottleneck [IB] is a technique for extracting information in some `input' random variable that is relevant for predicting some different 'output' random variable. IB works by encoding the input in a compressed 'bottleneck variable' from which the output can then be accurately decoded. IB can be difficult to compute in practice, and has been mainly developed for two limited cases: (1) discrete random variables with small state spaces, and (2) continuous random variables that are jointly Gaussian distributed (in which case the encoding and decoding maps are linear). We propose a method to perform IB in more general domains. Our approach can be applied to discrete or continuous inputs and outputs, and allows for nonlinear encoding and decoding maps. The method uses a novel upper bound on the IB objective, derived using a non-parametric estimator of mutual information and a variational approximation. We show how to implement the method using neural networks and gradient-based optimization, and demonstrate its performance on the MNIST dataset.

Why do we need Machine Learning? Machine learning is needed for tasks that are too complex for humans to code directly. Some tasks are so complex that it is impractical, if not impossible, for humans to work out all of the nuances and code for them explicitly. So instead, we provide a large amount of data to a machine learning algorithm and let the algorithm work it out by exploring that data and searching for a model that will achieve what the programmers have set it out to achieve. Let's look at these 2 examples: Then comes the Machine Learning Approach: Instead of writing a program by hand for each specific task, we collect lots of examples that specify the correct output for a given input. A machine learning algorithm then takes these examples and produces a program that does the job.

MACHINES are getting to know each other better. An artificial intelligence, developed by Google-owned research firm DeepMind, can now pass an important psychological assessment that most children only develop the skills to pass at around age 4. Its aptitude in this key theory of mind test may lead to AIs that are more human-like. Most humans regularly think about other people's desires, beliefs or intentions.

Summary: This is the second in our "Off the Beaten Path" series looking at innovators in machine learning who have elected strategies and methods outside of the mainstream. In this article we look at Numenta's unique approach to scalar prediction and anomaly detection based on their own brain research. Numenta, the machine intelligence company founded in 2005 by Jeff Hawkins of Palm Pilot fame might well be the poster child for'off the beaten path'. More a research laboratory than commercial venture, Hawkins has been pursuing a strong-AI model of computation that will at once directly model the human brain, and as a result be a general purpose solution to all types of machine learning problems. After swimming against the tide of the'narrow' or'weak' AI approaches represented by deep learning's CNNs and RNN/LSTMs his bet is starting to pay off.

According to the American Cancer Society, cancer kills more than 8 million people each year. Early detection can boost survival rates. Researchers and clinicians are feverishly exploring avenues to provide early and accurate diagnoses, as well as more targeted treatments. Blood screenings are used to detect many types of cancers, including liver, ovarian, colon and lung cancers. Current blood screening methods typically rely on affixing biochemical labels to cells or biomolecules.

I like to refer to the input layer as being "on the bottom" rather than at the far left as in this image. When viewing it my way, the low-to-high dimension we use in my rotated version of the above can be mentally mapped to a low-to-high stack of abstraction levels; I'm not the only one using this dimension this way. I hope this rotation isn't too confusing. We can see that there is an obvious data Reduction and an obvious complexity Reduction. Can we determine whether the system is also performing what I'd like to call "Epistemic Reduction": Is it reducing away that which is unimportant, and if so, how does it accomplish this? How does an operator in a Deep Learning stack know what makes something important (Salient)? A pure data "reduction" of sorts could be accomplished by compression schemes or even random deletion.

Over 60 years ago, in what is today southern Kazakhstan, the Soviet Union launched the world's first artificial satellite into Earth's orbit. The launch of Sputnik 1 provoked a panic in the United States that catalyzed a flurry of investment and research, which ultimately put man on the moon. China's Sputnik moment came in March 2016, when AlphaGo, an artificial intelligence program developed by Google's DeepMind, defeated South Korean Go master Lee Sedol. As The New York Times reported, AlphaGo's victory had a profound impact on politicians in China, one that spurred increased commitment to Beijing's effort to rule AI. In July 2017, China's State Council published an ambitious policy blueprint calling for the country to become "the world's primary AI innovation center" by 2030.