In a quaint Regency-era office overlooking London's Russell Square, I cofounded a company called DeepMind with two friends, Demis Hassabis and Shane Legg, in the summer of 2010. Our goal, one that still feels as ambitious and crazy and hopeful as it did back then, was to replicate the very thing that makes us unique as a species: our intelligence. To achieve this, we would need to create a system that could imitate and then eventually outperform all human cognitive abilities, from vision and speech to planning and imagination, and ultimately empathy and creativity. Since such a system would benefit from the massively parallel processing of supercomputers and the explosion of vast new sources of data from across the open web, we knew that even modest progress toward this goal would have profound societal implications. It certainly felt pretty far-out at the time.

Several test function suites are being used for numerical benchmarking of multiobjective optimization algorithms. While they have some desirable properties, like well-understood Pareto sets and Pareto fronts of various shapes, most of the currently used functions possess characteristics that are arguably under-represented in real-world problems. They mainly stem from the easier construction of such functions and result in improbable properties such as separability, optima located exactly at the boundary constraints, and the existence of variables that solely control the distance between a solution and the Pareto front. Here, we propose an alternative way to constructing multiobjective problems-by combining existing single-objective problems from the literature. We describe in particular the bbob-biobj test suite with 55 bi-objective functions in continuous domain, and its extended version with 92 bi-objective functions (bbob-biobj-ext). Both test suites have been implemented in the COCO platform for black-box optimization benchmarking. Finally, we recommend a general procedure for creating test suites for an arbitrary number of objectives. Besides providing the formal function definitions and presenting their (known) properties, this paper also aims at giving the rationale behind our approach in terms of groups of functions with similar properties, objective space normalization, and problem instances. The latter allows us to easily compare the performance of deterministic and stochastic solvers, which is an often overlooked issue in benchmarking.

Every day, millions of people gather on online game servers to collaborate in real time toward shared goals. What may seem like frivolous activity may, when investigated more closely, provide revolutionary opportunities to advance the science of teamwork. Teamwork is an important part of modern society, however, collaboration between people is often made difficult due to differing ideals, opinions, and personality types. We leverage a popular self-hosted multiplayer online game environment to design a framework for developing and deploying tasks that elicit different kinds of teamwork. We propose to use these to capture fine-scale details of individual and group performance across environments. The game in which we implement this system, Minecraft, is ideal because it is heavily modifiable and already enjoys a vast user base of surprising gender, age, and ethnic diversity. We heavily modify the game in order to introduce new mechanics that facilitate collaboration, monitor activity, and manipulate group composition, all to provide the groundwork for deeper quantitative insights into effective teams.