We consider an agent's uncertainty about its environment and the problem of generalizing this uncertainty across observations. Specifically, we focus on the problem of exploration in non-tabular reinforcement learning. Drawing inspiration from the intrinsic motivation literature, we use density models to measure uncertainty, and propose a novel algorithm for deriving a pseudo-count from an arbitrary density model. This technique enables us to generalize count-based exploration algorithms to the non-tabular case. We apply our ideas to Atari 2600 games, providing sensible pseudo-counts from raw pixels. We transform these pseudo-counts into intrinsic rewards and obtain significantly improved exploration in a number of hard games, including the infamously difficult Montezuma's Revenge.

Vyacheslav is a PhD candidate at the Oxford Internet Institute. His research uses social psychology and machine learning to understand networks of people and networks of ideas. Imagine a typical day in 2020: Your personal AI assistant wakes you up with a friendly greeting before preparing your favorite breakfast. During your morning workout, it plays new songs that perfectly match your musical tastes. For your driverless commute to work, it has pre-selected a few articles based on the duration of your commute and what you've read in the past.

I describe here the projects that I worked on, as well as career progress, starting 25 years ago as a PhD student in statistics, until today, and the transformation from statistician to data scientist that occurred slowly and started more than 20 years ago. This also illustrates many applications of data science, most are still active. My interest in mathematics started when I was 7 or 8, I remember being fascinated by the powers of 2 in primary school, and later purchasing cheap russian math books (Mir publisher) translated in French, for my entertainement. In high school, I participated in the mathematical olympiads, and did my own math research during math classes, rather than listening to the very boring lessons. When I attended college, I stopped showing up in the classroom altogether - afterall, you could just read the syllabus, memorize the material before the exam and regurgitate it at the exam.

Analyses performed using Spark of brain activity in a larval zebrafish: embedding dynamics of whole-brain activity into lower-dimensional trajectories. The growth of data volumes in industry and research poses tremendous opportunities, as well as tremendous computational challenges. As data sizes have outpaced the capabilities of single machines, users have needed new systems to scale out computations to multiple nodes. As a result, there has been an explosion of new cluster programming models targeting diverse computing workloads.1,4,7,10 At first, these models were relatively specialized, with new models developed for new workloads; for example, MapReduce4 supported batch processing, but Google also developed Dremel13 for interactive SQL queries and Pregel11 for iterative graph algorithms. In the open source Apache Hadoop stack, systems like Storm1 and Impala9 are also specialized. Even in the relational database world, the trend has been to move away from "one-size-fits-all" systems.18 Unfortunately, most big data applications need to combine many different processing types. The very nature of "big data" is that it is diverse and messy; a typical pipeline will need MapReduce-like code for data loading, SQL-like queries, and iterative machine learning. Specialized engines can thus create both complexity and inefficiency; users must stitch together disparate systems, and some applications simply cannot be expressed efficiently in any engine. In 2009, our group at the University of California, Berkeley, started the Apache Spark project to design a unified engine for distributed data processing. Spark has a programming model similar to MapReduce but extends it with a data-sharing abstraction called "Resilient Distributed Datasets," or RDDs.25 Using this simple extension, Spark can capture a wide range of processing workloads that previously needed separate engines, including SQL, streaming, machine learning, and graph processing2,26,6 (see Figure 1).

When robots inevitably take over our planet, as the dystopian vision of science fiction writers foretells, we'll lose our jobs, our freedom, our humanity. But take comfort in one thing the robots will provide for us lowly carbon-based life-forms: artisanal pizza. They're already making it in a commercial kitchen in the heart of Silicon Valley: Two robots named Pepe and Giorgio squirt sauce on dough, and another robot, Marta, spreads it. A robotic arm named Bruno puts the pizza in the oven. They don't operate independently from humans yet -- two or three people still load the dough onto the conveyor and sprinkle cheese and toppings -- but Zume Pizza in Mountain View expects to be fully automated by spring, delivering made-to-order, customizable pizzas in as little as seven minutes.

Sitting in a studio in Kensington, London, the designer Sebastian Conran walks me through a worst-case scenario. "Basically, what it's looking for is a break in routine," he explains, pointing to a drawing of an elderly woman, collapsed on the floor of her home. The e-sensor in the room notices that you've fallen over. MiRo is a robotic dog. There is an early model close to where we are sitting.

Samsung Electronics Co. plans to equip its next Galaxy S smartphones with a Siri-like digital assistant, seeking to make a strong comeback after the global debacle that precipitated the death of its flawed Note 7 lineup. Samsung, which last month acquired U.S.-based artificial-intelligence software company Viv Labs Inc., said the Galaxy S8 slated for next year will come with AI-enabled features "significantly differentiated" from those of Apple Inc.'s Siri or Google, executive vice president Rhee In-jong told reporters. Those services now offer up potentially useful information from the weather to flight times based on user activity. "It will be significantly differentiated from the current services we see in the market now," Rhee said of the S8's features. He said last month the company plans to embed Viv's technology in other electronics and home appliances beyond phones. Global technology players are vying to build devices attuned to users' lifestyles and daily behavior.

With GPS technology at our fingertips and Siri available to navigate our every turn, paper maps have been rendered obsolete. But that is exactly what Japanese photographer Sohei Nishino has created with his series of collaged "Diorama Maps," which went on view Friday at the San Francisco Museum of Modern Art. After initial research of a city, Nishino spends up to three months walking it while snapping photos in black and white. "He climbs up to rooftops and high floors of buildings to get a multifaceted bird's-eye view," curator Corey Keller said. "These vantage points give him an alternative perspective of the city."

Disclosure: Organizers of the "France is AI" conference paid travel expenses for the reporter to attend and moderate several sessions at the event. Tucked into a courtyard in central Paris, the 35 employees of Snips are hunched over their computers trying to put the finishing touches on a new version of the company's artificial intelligence app for smartphones. The company is packed full of big brains, many of them products of France's leading universities and a culture that is historically strong in mathematics. And they're not afraid to let you know it. Etched into winding wooden staircases that lead to Snips offices are a series of math puzzles that job recruits are asked to solve as they work their way upstairs. Snips' app wants to scan all the data across the apps on your smartphones to deliver insight about you and eventually become a hyper-smart personal assistant.

Britain's manufacturing sector has just posted its strongest growth in over two years. Latest figures show that export orders have increased at their fastest rate since January 2014 and factories have also taken on more workers, with employment rising for the second consecutive month. However, the export benefits of a weakened pound will not last forever, and so, as the manufacturing sector continues to evolve, this year's FT Future of Manufacturing Summit looked at how big data analytics, advanced robotics, the Internet of Things (IoT) and additive manufacturing are shaping the economics of production and distribution within the sector. With the opportunities big data brings referred to as the Fourth Industrial Revolution in manufacturing, the estimated £57bn boost to the industry over the next five years will be driven by gains in efficiency through the use of big data analytics. The winners will be those who can adapt, embrace technologies and respond to new demands.