We introduce a novel approach for parallelizing MCMC inference in models with spatially determined conditional independence relationships, for which existing techniques exploiting graphical model structure are not applicable. Our approach is motivated by a model of seismic events and signals, where events detected in distant regions are approximately independent given those in intermediate regions. We perform parallel inference by coloring a factor graph defined over regions of latent space, rather than individual model variables. Evaluating on a model of seismic event detection, we achieve significant speedups over serial MCMC with no degradation in inference quality.

In this paper, we study inexact damped Newton method implemented in a distributed environment. We are motivated by the original DiSCO algorithm [Communication-Efficient Distributed Optimization of Self-Concordant Empirical Loss, Yuchen Zhang and Lin Xiao, 2015].We show that this algorithm may not scale well and propose algorithmic modifications which lead to fewer communications and better load-balancing between nodes. Those modifications lead to a more efficient algorithm with better scaling. This was made possibly by introducing our new pre-conditioner which is specially designed so that the preconditioning step can be solved exactly and efficiently.Numerical experiments for minimization of regularized empirical loss with a 273GB instance shows the efficiency of proposed algorithm.

Two popular approaches for distributed training of SVMs on big data are parameter averaging and alternating direction method of multipliers (ADMM). Parameter averaging is efficient but suffers from loss of accuracy with increase in number of partitions, while ADMM in the feature space is accurate but suffers from slow convergence. In this paper, we report a hybrid approach called weighted parameter averaging (WPA), which optimizes the regularized hinge loss with respect to weights on parameters. The problem is shown to be same as solving SVM in a projected space. We also demonstrate an O(1/N) stability bound on final hypothesis given by WPA, using novel proof techniques. Experimental results on a variety of toy and real world datasets show that our approach is significantly more accurate than parameter averaging for high number of partitions. It is also seen the proposed method enjoys much faster convergence compared to ADMM in feature space.

Humans develop a common sense of style compatibility between items based on their attributes. We seek to automatically answer questions like "Does this shirt go well with that pair of jeans?" In order to answer these kinds of questions, we attempt to model human sense of style compatibility in this paper. The basic assumption of our approach is that most of the important attributes for a product in an online store are included in its title description. Therefore it is feasible to learn style compatibility from these descriptions. We design a Siamese Convolutional Neural Network architecture and feed it with title pairs of items, which are either compatible or incompatible. Those pairs will be mapped from the original space of symbolic words into some embedded style space. Our approach takes only words as the input with few preprocessing and there is no laborious and expensive feature engineering.

In this paper, we propose an efficient transfer leaning methods for training a personalized language model using a recurrent neural network with long short-term memory architecture. With our proposed fast transfer learning schemes, a general language model is updated to a personalized language model with a small amount of user data and a limited computing resource. These methods are especially useful for a mobile device environment while the data is prevented from transferring out of the device for privacy purposes. Through experiments on dialogue data in a drama, it is verified that our transfer learning methods have successfully generated the personalized language model, whose output is more similar to the personal language style in both qualitative and quantitative aspects.

Recurrent Neural Networks (RNNs) and their variants (suchas LSTMs and GRUs) have been remarkably successful atmachine-learning tasks on diverse kinds of sequential data(e.g. text, time-series, etc.). However, training of RNNs con-tinue to be a challenge due to difficulties stemming from regu-larization and the highly non-convex optimizations involved.In this paper, we propose to regularize training of RNNs byencouraging higher decorrelation in the hidden representa-tions. The cost function is devised to minimize non-diagonalelements of the correlation matrix computed over the hid-den representations of RNNs, along with the usual trainingaccuracy term; thereby penalizing redundancy in the learnedmodel. Furthermore, we propose to utilize the idea of decor-relating representations in learning an ensemble of RNNs,in order to maximize diversity in the resulting models; thusenforcing every individual network of the ensemble to gainabilities that are complementary to the ensemble. Extensiveexperiments are presented on various datasets with differentarchitectures of RNNs. Results are offered for multiple tasksand show that the proposed methods yield a significant im-provement; when compared with the state-of-the-art methods.

Collaborative filtering (CF) with implicit feedback is a successful method for recommending items to users, which does not require a knowledge of the items or users. CF methods can be mainly classified into two categories. One is point-wise regression based and the other is pair-wise ranking based, where the latter one only tries to find out the items that users prefer while ignores the items that users dislike, and usually gives out a better recommended item list. The performance of CF-based methods degrades significantly when the feedback information is sparse. To address the problem, many kinds of auxiliary information have been utilized such as users’ reviews on items, items’ content and description information, price, brands. In this paper we utilize a weighted recursive autoencoder (RAE) to extract useful features from several heterogeneous auxiliary information and tightly couple the weighted RAE with a pair-wise ranking based CF method. Analysis of the hyperparameters illustrates that auxiliary information from different sources is indeed able to benefit our model. Empirical experiments on six real world datasets show that our method outperforms other state-of-the-art methods.

Hand pose estimation plays an important role in human-computer interaction and virtual reality. In this paper, we present a regression framework to estimate 3D hand pose using depth image. Different from the previous methods, we propose a new method that has three key aspects: first, performance of system can be improved by setting up the better initial images using feature extraction via Convolution Neural Network (CNN); secondly, the error of joint position is estimated by dividing the dataset into groups of gesture type; thirdly, accuracy can be improved by learning the residual intensity of depth image by updating the residual of 3D joint coordinates constantly. It is noticed that importance of categorizing hand poses by gesture in computing the joint positions has been underestimated. Experimental evaluation with a public dataset A*STAR shows that our method produces low error of hand pose estimation and has more potential for the future work of the hand pose estimation.

Many prediction problems can be phrased as inferences over local neighborhoods of graphs. The graph represents the interaction between entities, and the neighborhood of each entity contains information that allows the inferences or predictions. We present an approach for applying machine learning directly to such graph neighborhoods, yielding predictions for graph nodes on the basis of the structure of their local neighborhood and the features of the nodes in it. Our approach allows predictions to be learned directly from examples, bypassing the step of creating and tuning an inference model or summarizing the neighborhoods via a fixed set of hand-crafted features. The approach is based on a multi-level architecture built from Long Short-Term Memory neural nets (LSTMs); the LSTMs learn how to summarize the neighborhood from data. We demonstrate the effectiveness of the proposed technique on a synthetic example and on real-world data related to crowdsourced grading, Bitcoin transactions, and Wikipedia edit reversions.

The Strong Stackelberg Equilibrium (SSE) has drawn extensive attention recently in several security domains, which optimizes the defender's random allocation of limited security resources. However, the SSE concept neglects the advantage of defender's strategic revelation of her private information, and overestimates the observation ability of the adversaries. In this paper, we overcome these restrictions and analyze the tradeoff between strategic secrecy and commitment in security games. We propose a Disguised-resource Security Game (DSG) where the defender strategically disguises some of her resources. We compare strategic information revelation with public commitment and formally show that they have different advantages depending the payoff structure. To compute the Perfect Bayesian Equilibrium (PBE), several novel approaches are provided, including basic MILP formulations with mixed defender strategy and compact representation, a novel algorithm based on support set enumeration, and an approximation algorithm for epsilon-PBE. Extensive experimental evaluation shows that both strategic secrecy and Stackelberg commitment are critical measures in security domain, and our approaches can solve PBE for realistic-sized problems with good enough and robust solution quality.