Problem Solving


Human insight remains essential to beat the bias of algorithms

#artificialintelligence

When it comes to bias and artificial intelligence, there is a common belief that algorithms are only as good as the numbers plugged into them. But the focus on algorithmic bias being concentrated entirely on data has meant we have ignored two aspects of this problem: the deep limitations of existing algorithms and, more importantly, the role of human problem solvers. Powerful as they may be, most of our algorithms only mine correlational relationships without understanding anything about them. My research has found that massive data sets on jobs, education and loans contain more spurious correlations than meaningful causal relationships. It is ludicrous to assume these algorithms will solve problems that we do not understand.



How LEGO Is Training The Scientists And Problem Solvers Of The Future

#artificialintelligence

Through play children (and adults) learn how to use their imaginations, to experiment with different ways of doing things. This might seem like it has relevance only for their self-development, but it's also through imagination and experimentation that the human race as a collective arrives at the solutions to its problems. As such, it's vital that we encourage children and people more generally to use their imaginations and to experiment, and it's to this end that LEGO, of all things, has an important role to play in nurturing the next generation of engineers, scientists and problem solvers. And we're not just talking about informal play with LEGO here, since one organization in particular has taken it upon itself to incorporate the famous Danish toy in competitions and workshops, all of which aim to instil a love for science and engineering in children. This organization is FIRST (For Inspiration and Recognition of Science and Technology), a not-for-profit public charity based in New Hampshire that works to inspire young people to pursue careers and education in STEM (science, technology, engineering and mathematics) subjects.


Rapid Deformable Object Detection using Dual-Tree Branch-and-Bound

Neural Information Processing Systems

In this work we use Branch-and-Bound (BB) to efficiently detect objects with deformable part models. Instead of evaluating the classifier score exhaustively over image locations and scales, we use BB to focus on promising image locations. The core problem is to compute bounds that accommodate part deformations; for this we adapt the Dual Trees data structure to our problem. We evaluate our approach using Mixture-of-Deformable Part Models. We obtain exactly the same results but are 10-20 times faster on average.


Divide-and-Conquer Matrix Factorization

Neural Information Processing Systems

This work introduces Divide-Factor-Combine (DFC), a parallel divide-and-conquer framework for noisy matrix factorization. DFC divides a large-scale matrix factorization task into smaller subproblems, solves each subproblem in parallel using an arbitrary base matrix factorization algorithm, and combines the subproblem solutions using techniques from randomized matrix approximation. Our experiments with collaborative filtering, video background modeling, and simulated data demonstrate the near-linear to super-linear speed-ups attainable with this approach. Moreover, our analysis shows that DFC enjoys high-probability recovery guarantees comparable to those of its base algorithm. Papers published at the Neural Information Processing Systems Conference.


Expressive Power and Approximation Errors of Restricted Boltzmann Machines

Neural Information Processing Systems

We present explicit classes of probability distributions that can be learned by Restricted Boltzmann Machines (RBMs) depending on the number of units that they contain, and which are representative for the expressive power of the model. We use this to show that the maximal Kullback-Leibler divergence to the RBM model with n visible and m hidden units is bounded from above by (n-1)-log(m 1). In this way we can specify the number of hidden units that guarantees a sufficiently rich model containing different classes of distributions and respecting a given error tolerance. Papers published at the Neural Information Processing Systems Conference.


A Divide-and-Conquer Method for Sparse Inverse Covariance Estimation

Neural Information Processing Systems

Even in the face of this high dimensionality, and with limited number of samples, recent work has shown this estimator to have strong statistical guarantees in recovering the true structure of the sparse inverse covariance matrix, or alternatively the underlying graph structure of the corresponding Gaussian Markov Random Field. Our proposed algorithm divides the problem into smaller sub-problems, and uses the solutions of the sub-problems to build a good approximation for the original problem. We derive a bound on the distance of the approximate solution to the true solution. Based on this bound, we propose a clustering algorithm that attempts to minimize this bound, and in practice, is able to find effective partitions of the variables. We further use the approximate solution, i.e., solution resulting from solving the sub-problems, as an initial point to solve the original problem, and achieve a much faster computational procedure.


Efficient Bayes-Adaptive Reinforcement Learning using Sample-Based Search

Neural Information Processing Systems

Bayesian model-based reinforcement learning is a formally elegant approach to learning optimal behaviour under model uncertainty, trading off exploration and exploitation in an ideal way. Unfortunately, finding the resulting Bayes-optimal policies is notoriously taxing, since the search space becomes enormous. In this paper we introduce a tractable, sample-based method for approximate Bayes-optimal planning which exploits Monte-Carlo tree search. Our approach outperformed prior Bayesian model-based RL algorithms by a significant margin on several well-known benchmark problems -- because it avoids expensive applications of Bayes rule within the search tree by lazily sampling models from the current beliefs. We illustrate the advantages of our approach by showing it working in an infinite state space domain which is qualitatively out of reach of almost all previous work in Bayesian exploration.


Modeling Conceptual Understanding in Image Reference Games

Neural Information Processing Systems

An agent who interacts with a wide population of other agents needs to be aware that there may be variations in their understanding of the world. Furthermore, the machinery which they use to perceive may be inherently different, as is the case between humans and machines. In this work, we present both an image reference game between a speaker and a population of listeners where reasoning about the concepts other agents can comprehend is necessary and a model formulation with this capability. We focus on reasoning about the conceptual understanding of others, as well as adapting to novel gameplay partners and dealing with differences in perceptual machinery. Our experiments on three benchmark image/attribute datasets suggest that our learner indeed encodes information directly pertaining to the understanding of other agents, and that leveraging this information is crucial for maximizing gameplay performance.


Learning search spaces for Bayesian optimization: Another view of hyperparameter transfer learning

Neural Information Processing Systems

Bayesian optimization (BO) is a successful methodology to optimize black-box functions that are expensive to evaluate. While traditional methods optimize each black-box function in isolation, there has been recent interest in speeding up BO by transferring knowledge across multiple related black-box functions. In this work, we introduce a method to automatically design the BO search space by relying on evaluations of previous black-box functions. We depart from the common practice of defining a set of arbitrary search ranges a priori by considering search space geometries that are learnt from historical data. This simple, yet effective strategy can be used to endow many existing BO methods with transfer learning properties.