Evolutionary Systems


How AI can help the move to a low-carbon future

#artificialintelligence

There's a dire need to speed the planet's shift to clean energy - and the power of Artificial Intelligence can help. The world has gone through a number of energy transformations – from wood to coal, then to oil, gas and (partly) nuclear. These shifts were gradual and contingent on economic conditions. Now major efforts are under way to reform the global energy sector to make it low-carbon, non-nuclear and climate-compatible. But, unlike the previous transformations, the ongoing restructuring process is driven by elevated awareness of the disastrous consequences of climate change.


Predictive Analytics World Industry 4.0 Munich Agenda

#artificialintelligence

Birds do not collide when they fly in flocks. We may wonder how they do not and how they flock in a self-organized and well-orchestrated movement. It is a collective intelligence that is encapsulated within the interactions between the birds and the environment. The cohesive self-organized movement of a biological swarm such as flocking birds is commonly studied. Such phenomena have had successful applications in robotics and autonomous vehicles, and it has attracted a renewed interest from the Artificial Intelligence and the Predictive Analytics communities.




Bias and Generalization in Deep Generative Models: An Empirical Study

Neural Information Processing Systems

In high dimensional settings, density estimation algorithms rely crucially on their inductive bias. Despite recent empirical success, the inductive bias of deep generative models is not well understood. In this paper we propose a framework to systematically investigate bias and generalization in deep generative models of images by probing the learning algorithm with carefully designed training datasets. By measuring properties of the learned distribution, we are able to find interesting patterns of generalization. We verify that these patterns are consistent across datasets, common models and architectures.


A no-regret generalization of hierarchical softmax to extreme multi-label classification

Neural Information Processing Systems

Extreme multi-label classification (XMLC) is a problem of tagging an instance with a small subset of relevant labels chosen from an extremely large pool of possible labels. Large label spaces can be efficiently handled by organizing labels as a tree, like in the hierarchical softmax (HSM) approach commonly used for multi-class problems. In this paper, we investigate probabilistic label trees (PLTs) that have been recently devised for tackling XMLC problems. We show that PLTs are a no-regret multi-label generalization of HSM when precision@$k$ is used as a model evaluation metric. Critically, we prove that pick-one-label heuristic---a reduction technique from multi-label to multi-class that is routinely used along with HSM---is not consistent in general.


Generalization Bounds and Consistency for Latent Structural Probit and Ramp Loss

Neural Information Processing Systems

We consider latent structural versions of probit loss and ramp loss. We show that these surrogate loss functions are consistent in the strong sense that for any feature map (finite or infinite dimensional) they yield predictors approaching the infimum task loss achievable by any linear predictor over the given features. We also give finite sample generalization bounds (convergence rates) for these loss functions. These bounds suggest that probit loss converges more rapidly. However, ramp loss is more easily optimized and may ultimately be more practical.


On the Generalization Ability of Online Strongly Convex Programming Algorithms

Neural Information Processing Systems

This paper examines the generalization properties of online convex programming algorithms when the loss function is Lipschitz and strongly convex. Our main result is a sharp bound, that holds with high probability, on the excess risk of the output of an online algorithm in terms of the average regret. This allows one to use recent algorithms with logarithmic cumulative regret guarantees to achieve fast convergence rates for the excess risk with high probability. The bound also solves an open problem regarding the convergence rate of {\pegasos}, a recently proposed method for solving the SVM optimization problem. Papers published at the Neural Information Processing Systems Conference.


Two-Layer Generalization Analysis for Ranking Using Rademacher Average

Neural Information Processing Systems

This paper is concerned with the generalization analysis on learning to rank for information retrieval (IR). In IR, data are hierarchically organized, i.e., consisting of queries and documents per query. Previous generalization analysis for ranking, however, has not fully considered this structure, and cannot explain how the simultaneous change of query number and document number in the training data will affect the performance of algorithms. In this paper, we propose performing generalization analysis under the assumption of two-layer sampling, i.e., the i.i.d. Such a sampling can better describe the generation mechanism of real data, and the corresponding generalization analysis can better explain the real behaviors of learning to rank algorithms.


Generalization Errors and Learning Curves for Regression with Multi-task Gaussian Processes

Neural Information Processing Systems

We provide some insights into how task correlations in multi-task Gaussian process (GP) regression affect the generalization error and the learning curve. We analyze the asymmetric two-task case, where a secondary task is to help the learning of a primary task. Within this setting, we give bounds on the generalization error and the learning curve of the primary task. Our approach admits intuitive understandings of the multi-task GP by relating it to single-task GPs. For the case of one-dimensional input-space under optimal sampling with data only for the secondary task, the limitations of multi-task GP can be quantified explicitly.