Forecasting a time series from multivariate predictors constitutes a challenging problem, especially using model-free approaches. Most techniques, such as nearest-neighbor prediction, quickly suffer from the curse of dimensionality and overfitting for more than a few predictors which has limited their application mostly to the univariate case. Therefore, selection strategies are needed that harness the available information as efficiently as possible. Since often the right combination of predictors matters, ideally all subsets of possible predictors should be tested for their predictive power, but the exponentially growing number of combinations makes such an approach computationally prohibitive. Here a prediction scheme that overcomes this strong limitation is introduced utilizing a causal pre-selection step which drastically reduces the number of possible predictors to the most predictive set of causal drivers making a globally optimal search scheme tractable. The information-theoretic optimality is derived and practical selection criteria are discussed. As demonstrated for multivariate nonlinear stochastic delay processes, the optimal scheme can even be less computationally expensive than commonly used sub-optimal schemes like forward selection. The method suggests a general framework to apply the optimal model-free approach to select variables and subsequently fit a model to further improve a prediction or learn statistical dependencies. The performance of this framework is illustrated on a climatological index of El Ni\~no Southern Oscillation.

In order to investigate the breast cancer prediction problem on the aging population with the grades of DCIS, we conduct a tree augmented naive Bayesian network experiment trained and tested on a large clinical dataset including consecutive diagnostic mammography examinations, consequent biopsy outcomes and related cancer registry records in the population of women across all ages. The aggregated results of our ten-fold cross validation method recommend a biopsy threshold higher than 2% for the aging population.

The computational complexity of kernel methods has often been a major barrier for applying them to large-scale learning problems. We argue that this barrier can be effectively overcome. In particular, we develop methods to scale up kernel models to successfully tackle large-scale learning problems that are so far only approachable by deep learning architectures. Based on the seminal work by Rahimi and Recht on approximating kernel functions with features derived from random projections, we advance the state-of-the-art by proposing methods that can efficiently train models with hundreds of millions of parameters, and learn optimal representations from multiple kernels. We conduct extensive empirical studies on problems from image recognition and automatic speech recognition, and show that the performance of our kernel models matches that of well-engineered deep neural nets (DNNs). To the best of our knowledge, this is the first time that a direct comparison between these two methods on large-scale problems is reported. Our kernel methods have several appealing properties: training with convex optimization, cost for training a single model comparable to DNNs, and significantly reduced total cost due to fewer hyperparameters to tune for model selection. Our contrastive study between these two very different but equally competitive models sheds light on fundamental questions such as how to learn good representations.

Over the past years Robust PCA has been established as a standard tool for reliable low-rank approximation of matrices in the presence of outliers. Recently, the Robust PCA approach via nuclear norm minimization has been extended to matrices with linear structures which appear in applications such as system identification and data series analysis. At the same time it has been shown how to control the rank of a structured approximation via matrix factorization approaches. The drawbacks of these methods either lie in the lack of robustness against outliers or in their static nature of repeated batch-processing. We present a Robust Structured Low-Rank Approximation method on the Grassmannian that on the one hand allows for fast re-initialization in an online setting due to subspace identification with manifolds, and that is robust against outliers due to a smooth approximation of the $\ell_p$-norm cost function on the other hand. The method is evaluated in online time series forecasting tasks on simulated and real-world data.

We present a Bayesian tensor factorization model for inferring latent group structures from dynamic pairwise interaction patterns. For decades, political scientists have collected and analyzed records of the form "country $i$ took action $a$ toward country $j$ at time $t$"---known as dyadic events---in order to form and test theories of international relations. We represent these event data as a tensor of counts and develop Bayesian Poisson tensor factorization to infer a low-dimensional, interpretable representation of their salient patterns. We demonstrate that our model's predictive performance is better than that of standard non-negative tensor factorization methods. We also provide a comparison of our variational updates to their maximum likelihood counterparts. In doing so, we identify a better way to form point estimates of the latent factors than that typically used in Bayesian Poisson matrix factorization. Finally, we showcase our model as an exploratory analysis tool for political scientists. We show that the inferred latent factor matrices capture interpretable multilateral relations that both conform to and inform our knowledge of international affairs.

A robot that evoked a human form paused in front of a door leading to a simulated nuclear power plant accident and inexplicably stood motionless. Suddenly, from the grandstands overlooking the scene, a group of schoolchildren began to chant: "Go Robot! What has long been thought of as a brave new world in which mobile robots freely move about in factories, towns and cities is now approaching. Robots will advance from the dull, dirty and dangerous work that they do today to take on a range of tasks, from rescue work to elder care in close contact with humans. Just as software robots such as Apple's Siri and Microsoft's Cortana have rapidly become useful personal assistants, physical robots will occupy a place in the near future. That is the world imagined by government officials and technologists at the Defense Advanced Research Projects Agency, the American military organization that is charged with the mission of avoiding a Sputnik-style technology threat to national security. Last weekend at the sprawling Los Angeles County Fairgrounds, Darpa concluded the Robotics Challenge, a two-year-long effort to jump start this next generation of smart and presumably helpful robots by offering a cash prize for the designers of a machine that could work in concert with human controllers in a hazardous environment. The $3.5 million competition was won by a South Korean team from the Korean Advanced Institute of Science and Technology. The technology may still seem far-fetched, but betting against the agency that has had a remarkably far-reaching effect on the modern world -- from funding the work that led to both the personal computer and the Internet, to setting expectations that self-driving vehicles are only a matter of years away -- might be a mistake. Darpa officials have taken pains to assure anyone who would listen that it was not primarily interested in designing Terminators, or killer robots. The agency is an arm of the Pentagon, and its futuristic robots are an example of what is described as a "dual use" technology that will have both military and civilian uses. Darpa, which is also known for pioneering the Internet surveillance system that was exposed last year by Edward J. Snowden, has, under its current director, Arati Prabhakar, expanded its watchfulness over the potential effect of the technologies it helps foster. In introducing a workshop for discussion on the effect of robotics held at the end of the challenge competition on Sunday, Dr. Prabhakar described the agency as being committed to a broader mission: "We work together to build the future of robots that can help extend the capabilities that we have and build the technologies that will aid humanity in the future.

We present a new, fully generative model of optical telescope image sets, along with a variational procedure for inference. Each pixel intensity is treated as a Poisson random variable, with a rate parameter dependent on latent properties of stars and galaxies. Key latent properties are themselves random, with scientific prior distributions constructed from large ancillary data sets. We check our approach on synthetic images. We also run it on images from a major sky survey, where it exceeds the performance of the current state-of-the-art method for locating celestial bodies and measuring their colors.

Cortana is taking the war of words to Siri's home turf, Amazon amazes us with its tax return, and we find out why the record industry has a love-hate relationship with YouTube. Microsoft is challenging Apple's voice-activated personal assistant Siri by bringing its own voice control system Cortana to iPhones, not to mention Android devices too. It's all part of the build-up to Windows 10, but will iPhone owners switch allegiance? Amazon is changing the way it operates in Europe to pay more taxes in individual countries, ending its much-criticised wheeze of exploiting the tax haven of Luxembourg. Tax doesn't have to be taxing.

Knowledge base (KB) completion adds new facts to a KB by making inferences from existing facts, for example by inferring with high likelihood nationality(X,Y) from bornIn(X,Y). Most previous methods infer simple one-hop relational synonyms like this, or use as evidence a multi-hop relational path treated as an atomic feature, like bornIn(X,Z) -> containedIn(Z,Y). This paper presents an approach that reasons about conjunctions of multi-hop relations non-atomically, composing the implications of a path using a recursive neural network (RNN) that takes as inputs vector embeddings of the binary relation in the path. Not only does this allow us to generalize to paths unseen at training time, but also, with a single high-capacity RNN, to predict new relation types not seen when the compositional model was trained (zero-shot learning). We assemble a new dataset of over 52M relational triples, and show that our method improves over a traditional classifier by 11%, and a method leveraging pre-trained embeddings by 7%.

We present Caffe con Troll (CcT), a fully compatible end-to-end version of the popular framework Caffe with rebuilt internals. We built CcT to examine the performance characteristics of training and deploying general-purpose convolutional neural networks across different hardware architectures. We find that, by employing standard batching optimizations for CPU training, we achieve a 4.5x throughput improvement over Caffe on popular networks like CaffeNet. Moreover, with these improvements, the end-to-end training time for CNNs is directly proportional to the FLOPS delivered by the CPU, which enables us to efficiently train hybrid CPU-GPU systems for CNNs.