Kernel alignment measures the degree of similarity between two kernels. In this paper, inspired from kernel alignment, we propose a new Linear Discriminant Analysis (LDA) formulation, kernel alignment LDA (kaLDA). We first define two kernels, data kernel and class indicator kernel. The problem is to find a subspace to maximize the alignment between subspace-transformed data kernel and class indicator kernel. Surprisingly, the kernel alignment induced kaLDA objective function is very similar to classical LDA and can be expressed using between-class and total scatter matrices. This can be extended to multi-label data. We use a Stiefel-manifold gradient descent algorithm to solve this problem. We perform experiments on 8 single-label and 6 multi-label data sets. Results show that kaLDA has very good performance on many single-label and multi-label problems.

Stochastic linear bandits are a natural and simple generalisation of finite-armed bandits with numerous practical applications. Current approaches focus on generalising existing techniques for finite-armed bandits, notably the optimism principle and Thompson sampling. While prior work has mostly been in the worst-case setting, we analyse the asymptotic instance-dependent regret and show matching upper and lower bounds on what is achievable. Surprisingly, our results show that no algorithm based on optimism or Thompson sampling will ever achieve the optimal rate, and indeed, can be arbitrarily far from optimal, even in very simple cases. This is a disturbing result because these techniques are standard tools that are widely used for sequential optimisation.

We propose a simple discrete time semi-supervised graph embedding approach to link prediction in dynamic networks. The learned embedding reflects information from both the temporal and cross-sectional network structures, which is performed by defining the loss function as a weighted sum of the supervised loss from past dynamics and the unsupervised loss of predicting the neighborhood context in the current network. Our model is also capable of learning different embeddings for both formation and dissolution dynamics. These key aspects contributes to the predictive performance of our model and we provide experiments with three real--world dynamic networks showing that our method is comparable to state of the art methods in link formation prediction and outperforms state of the art baseline methods in link dissolution prediction.

Factorization machines (FMs) are a supervised learning approach that can use second-order feature combinations even when the data is very high-dimensional. Unfortunately, despite increasing interest in FMs, there exists to date no efficient training algorithm for higher-order FMs (HOFMs). In this paper, we present the first generic yet efficient algorithms for training arbitrary-order HOFMs. We also present new variants of HOFMs with shared parameters, which greatly reduce model size and prediction times while maintaining similar accuracy. We demonstrate the proposed approaches on four different link prediction tasks.

Obtaining labels can be costly and time-consuming. Active learning allows a learning algorithm to intelligently query samples to be labeled for efficient learning. Fisher information ratio (FIR) has been used as an objective for selecting queries in active learning. However, little is known about the theory behind the use of FIR for active learning. There is a gap between the underlying theory and the motivation of its usage in practice. In this paper, we attempt to fill this gap and provide a rigorous framework for analyzing existing FIR-based active learning methods. In particular, we show that FIR can be asymptotically viewed as an upper bound of the expected variance of the log-likelihood ratio. Additionally, our analysis suggests a unifying framework that not only enables us to make theoretical comparisons among the existing querying methods based on FIR, but also allows us to give insight into the development of new active learning approaches based on this objective.

In analyzing high-dimensional models, sparsity of the model parameter is a common but often undesirable assumption. In this paper, we study the following two-sample testing problem: given two samples generated by two high-dimensional linear models, we aim to test whether the regression coefficients of the two linear models are identical. We propose a framework named TIERS (short for TestIng Equality of Regression Slopes), which solves the two-sample testing problem without making any assumptions on the sparsity of the regression parameters. TIERS builds a new model by convolving the two samples in such a way that the original hypothesis translates into a new moment condition. A self-normalization construction is then developed to form a moment test. We provide rigorous theory for the developed framework. Under very weak conditions of the feature covariance, we show that the accuracy of the proposed test in controlling Type I errors is robust both to the lack of sparsity in the features and to the heavy tails in the error distribution, even when the sample size is much smaller than the feature dimension. Moreover, we discuss minimax optimality and efficiency properties of the proposed test. Simulation analysis demonstrates excellent finite-sample performance of our test. In deriving the test, we also develop tools that are of independent interest. The test is built upon a novel estimator, called Auto-aDaptive Dantzig Selector (ADDS), which not only automatically chooses an appropriate scale of the error term but also incorporates prior information. To effectively approximate the critical value of the test statistic, we develop a novel high-dimensional plug-in approach that complements the recent advances in Gaussian approximation theory.

Google seems to have taken another step forward with their progress in artificial intelligence as their new AI program can now navigate the London Underground system without repetitive feeding of data. Most AI programs can do the same but the difference with the new Google AI agent is that it can learn the ropes in just one try. In addition, the same program also has the capability to answer several questions regarding a family tree. Google DeepMind researchers developed the program without having to pre-program it to know what and how to learn. Once the map of the London Underground subway was given, it took care of the rest.

This post is aimed at artists and other creative people who are interested in a survey of recent developments in machine learning research that intersect with art and culture. If you've been following ML research recently, you might find some of the experiments interesting but will want to skip most of the explanations. The first AI that left me speechless was a chatbot named MegaHAL. It turns out MegaHAL was basically sleight of hand, picking a single word from your input and using a technique called Markov chains to iteratively guess the most likely words that would precede and follow based on a large corpus of example text (not unlike some Dada word games). But reading these transcripts in high school had a big effect on how I saw computers, and my interest in AI even affected where I applied to college.

When researchers at the University of Pittsburgh Medical Center blindfolded a paralyzed man whose was linked to a robotic hand, he could successfully identify which fingers were being touched 84 percent of the time. Mind-controlled robot arms can now generate feelings of touch, based on new research from the University of Pittsburgh Medical Center. The study, published today in Science Translational Medicine, represents a first for brain-computer interfaces and fulfills a major stage in creating robotic prosthetic arms for tetraplegics that can hold objects. "One of the reasons providing sensation is really important is when you reach out to pick something up, it's that sense of touch that allows you to hold the object properly," Robert Gaunt, the project's leader and a physical medicine and rehabilitation researcher at Pitt, told the NewsHour. That's because to touch an object like an apple, your brain requires two things: movement and feeling.

Do you think if we ever do develop AI with true intelligence, it will inevitably come with the kinds of problems the show seems poised to explore: robots with growing self-awareness and a desire to act in a way that goes against their programming? You would think that those challenges would very likely come up. We're talking about something that has never been done and may never be done. There is definitely a group of very smart people who are focused on that and, in particular, are very concerned about the potential existential threat from artificial intelligence, if we ever do build machines that can think for themselves and make decisions for themselves.