We propose a method for recovering the structure of a sparse undirected graphical model when very few samples are available. The method decides about the presence or absence of bonds between pairs of variable by considering one pair at a time and using a closed form formula, analytically derived by calculating the posterior probability for every possible model explaining a two body system using Jeffreys prior. The approach does not rely on the optimization of any cost functions and consequently is much faster than existing algorithms. Despite this time and computational advantage, numerical results show that for several sparse topologies the algorithm is comparable to the best existing algorithms, and is more accurate in the presence of hidden variables. We apply this approach to the analysis of US stock market data and to neural data, in order to show its efficiency in recovering robust statistical dependencies in real data with non-stationary correlations in time and/or space.

We introduce a new neural architecture to learn the conditional probability of an output sequence with elements that are discrete tokens corresponding to positions in an input sequence. Such problems cannot be trivially addressed by existent approaches such as sequence-to-sequence and Neural Turing Machines, because the number of target classes in each step of the output depends on the length of the input, which is variable. Problems such as sorting variable sized sequences, and various combinatorial optimization problems belong to this class. Our model solves the problem of variable size output dictionaries using a recently proposed mechanism of neural attention. It differs from the previous attention attempts in that, instead of using attention to blend hidden units of an encoder to a context vector at each decoder step, it uses attention as a pointer to select a member of the input sequence as the output. We call this architecture a Pointer Net (Ptr-Net). We show Ptr-Nets can be used to learn approximate solutions to three challenging geometric problems -- finding planar convex hulls, computing Delaunay triangulations, and the planar Travelling Salesman Problem -- using training examples alone. Ptr-Nets not only improve over sequence-to-sequence with input attention, but also allow us to generalize to variable size output dictionaries. We show that the learnt models generalize beyond the maximum lengths they were trained on. We hope our results on these tasks will encourage a broader exploration of neural learning for discrete problems.

ABSTRACT A new type of End-to-End system for text-dependent speaker verification is presented in this paper. Previously, using the phonetic/speaker discriminative DNNs as feature extractors for speaker verification has shown promising results. The extracted frame-level (DNN bottleneck, posterior or d-vector) features are equally weighted and aggregated to compute an utterance-level speaker representation (d-vector or i-vector). In this work we use speaker discriminative CNNs to extract the noise-robust frame-level features. These features are then combined to form an utterance-level speaker vector through an attention mechanism. The proposed attention model takes the speaker discriminative information and the phonetic information to learn the weights. The whole system, including the CNN and attention model, is joint optimized using an end-to- end criterion. The algorithm can automatically select the most similar impostor for each target speaker to train the network. We demonstrated the effectiveness of the proposed end-to-end system on Windows 10 "Hey Cortana" speaker verification task. Index Terms-- speaker verification, end-to-end training, attention model, deep learning, CNN 1. INTRODUCTION Speaker verification (SV) is a binary classification problem in which a person's identity is verified based on his/her voice.

To model categorical response variables given their covariates, we propose a permuted and augmented stick-breaking (paSB) construction that one-to-one maps the observed categories to randomly permuted latent sticks. This new construction transforms multinomial regression into regression analysis of stick-specific binary random variables that are mutually independent given their covariate-dependent stick success probabilities, which are parameterized by the regression coefficients of their corresponding categories. The paSB construction allows transforming an arbitrary cross-entropy-loss binary classifier into a Bayesian multinomial one. Specifically, we parameterize the negative logarithms of the stick failure probabilities with a family of covariate-dependent softplus functions to construct nonparametric Bayesian multinomial softplus regression, and transform Bayesian support vector machine (SVM) into Bayesian multinomial SVM. These Bayesian multinomial regression models are not only capable of providing probability estimates, quantifying uncertainty, and producing nonlinear classification decision boundaries, but also amenable to posterior simulation. Example results demonstrate their attractive properties and appealing performance.

We propose a methodology for testing linear hypothesis in high-dimensional linear models. The proposed test does not impose any restriction on the size of the model, i.e. model sparsity or the loading vector representing the hypothesis. Providing asymptotically valid methods for testing general linear functions of the regression parameters in high-dimensions is extremely challenging -- especially without making restrictive or unverifiable assumptions on the number of non-zero elements. We propose to test the moment conditions related to the newly designed restructured regression, where the inputs are transformed and augmented features. These new features incorporate the structure of the null hypothesis directly. The test statistics are constructed in such a way that lack of sparsity in the original model parameter does not present a problem for the theoretical justification of our procedures. We establish asymptotically exact control on Type I error without imposing any sparsity assumptions on model parameter or the vector representing the linear hypothesis. Our method is also shown to achieve certain optimality in detecting deviations from the null hypothesis. We demonstrate the favorable finite-sample performance of the proposed methods, via a number of numerical and a real data example.

Autoregression is a time series model that uses observations from previous time steps as input to a regression equation to predict the value at the next time step. It is a very simple idea that can result in accurate forecasts on a range of time series problems. In this tutorial, you will discover how to implement an autoregressive model for time series forecasting with Python. Autoregression Models for Time Series Forecasting With Python Photo by Umberto Salvagnin, some rights reserved. A regression model, such as linear regression, models an output value based on a linear combination of input values.

Gigaom, the leader in emerging technology research, today announced that it is holding its first annual competition for the AI start-up that delivers the highest ROI to corporate customers. The GAIN competition coincides with the annual conference Gigaom AI Now held in San Francisco, CA, February 15-16th 2017. AI startups are leading the way for bringing positive impacts of AI to many of the world's long-term challenges. "Record number of investments are being made in early stage AI start-ups. The competition will identify which new ventures can deliver the highest ROI to businesses today," explained David Hehman, Gigaom's Start-up & VC editor.

From Amazon and Facebook to Google and Microsoft, leaders of the world's most influential technology firms are highlighting their enthusiasm for Artificial Intelligence (AI). While there is growing interest in AI, the field is understood mainly by specialists. Our goal for this primer is to make this important field accessible to a broader audience. We'll begin by explaining the meaning of'AI' and key terms including'machine learning'. We'll illustrate how one of the most productive areas of AI, called'deep learning', works.

When it comes to robots replacing humans, we might think we have the upper hand since we're the ones who build and program them but that's not neccesarily the case anymore. Google is taking a different approach to training its robots – it's letting them teach each other. Researchers at Google have released a report showing how they connected 14 robotic arms together and used convolutional neural networks to let them teach themselves how to pick things up. The approach mimics how young children learn between the ages of one and four years old, and is essentially helping the robots to develop reliable hand-eye coordination. Typically, a robot would be programmed to carry out specific tasks, but this method shows how they can learn through trial-and-error in combination with a neural network – the same way a child learns how to do something by watching other people.

This tutorial introduces machine learning and data science concepts for developers. On the web, we already have many excellent resources for learning data science, however, the sheer amount of material can, in itself, be daunting. This is based on my insights from the Enterprise AI course and also the Data Science for IoT course which I teach at Oxford University. I also address a broader question: Which maths and stats techniques do you need for data science? A knowledge of algorithms (maths and stats) is the main differentiator between traditional programming and analytics-based programming. Having said that, it helps to start with programming and approach the maths (initially) through APIs and libraries.