One of the key challenges of artificial intelligence is to learn models that are effective in the context of planning. In this document we introduce the predictron architecture. The predictron consists of a fully abstract model, represented by a Markov reward process, that can be rolled forward multiple "imagined" planning steps. Each forward pass of the predictron accumulates internal rewards and values over multiple planning depths. The predictron is trained end-to-end so as to make these accumulated values accurately approximate the true value function. We applied the predictron to procedurally generated random mazes and a simulator for the game of pool. The predictron yielded significantly more accurate predictions than conventional deep neural network architectures.

This is the first part or a series of posts. To have a short introduction, read my Intro post. It is not rare for technology and engineering to take inspiration from nature's great designs. In this post, I will talk about genetic or evolutionary algorithms, their role in robotics, and more widely in computer science. Evolutionary algorithms are inspired by the natural process of evolution and natural selection.

Morgan Stanley is on a mission to combine the best of machine learning, predictive analytics and workflow technology with the human touch provided by its 16,000 financial advisers. Despite the popularity of robo-advisers, clients still want a human being to help them navigate complex life questions, according to Naureen Hassan, chief digital officer for wealth management at Morgan Stanley. Recent studies show that 82% of millennials who work with a financial adviser want more time with that adviser, not less, Hassan said. At the same time, clients also want a simple mobile app to keep track of everything when these decisions have been made. "I don't want to be forced into in-person meetings or shipping and signing a bunch of paperwork," Hassan said.

Machines are becoming more intelligent every year thanks to advances being made by companies like Google, Facebook, Microsoft, and many others. AI agents, as they're sometimes known, can already beat us at complex board games like Go, and they're becoming more competent in a range of other areas. Now a London artificial-intelligence research lab owned by Google has carried out a study to make sure that we can pull the plug on self-learning machines when we want to. DeepMind, bought by Google for a reported 400 million pounds -- about $580 million -- in 2014, teamed up with scientists at the University of Oxford to find a way to make sure that AI agents don't learn to prevent, or seek to prevent, humans from taking control. The paper -- "Safely Interruptible Agents PDF," published on the website of the Machine Intelligence Research Institute (MIRI) -- was written by Laurent Orseau, a research scientist at Google DeepMind, Stuart Armstrong at Oxford University's Future of Humanity Institute, and several others.

Apple's eagerly anticipated new iPhone will go on sale in September - but in very limited quantities, the latest forecast for the firm has claimed. J.P. Morgan Apple analyst Rod Hall believes the new handset will be called the'iPhone Pro', and will sell for $1100, according to 9to5Mac. However, the report warns that supply will be extremely limited at launch and for its first few months on sale. J.P. Morgan Apple analyst Rod Hall believes the new handset will be called the'iPhone Pro', and will sell for $1100 - but will only be available in limited quantities when it is announced in September. Multiple claims say the new OLED 5.8-inch iPhone, which is expected to have a bezel-less screen, may not be available until November.

Artificial intelligence software combined with a robotic harness could help spinal injury and stroke patients walk again. Rehabilitation programs for spinal cord injuries or strokes usually have patients walk on treadmills at a steady pace while harnesses support their weight to varying degrees. In the new study, researchers sought to develop a system that better mimicked the conditions that people might experience during everyday life, where they would have to move in more than one direction and vary their gaits. "The idea is to provide the most appropriate environment for patients to be active during training," says study co-author Grégoire Courtine, a neuroscientist at the Swiss Federal Institute of Technology Lausanne. "The goal of this rehabilitation is to have patients repeat natural activities for an extended amount of time." The scientists developed a robotic harness that uses cables to control the amount of upward and forward force that patients feel while also permitting them to walk forwards, backwards, and side to side.

The first voice you hear on the recording is mine. "Here we are," I say. My tone is cheerful, but a catch in my throat betrays how nervous I am. "Esquire," a second voice on the recording chimes in, and this one word--delivered as a winking parody of lawyerly pomposity--immediately puts me more at ease. The speaker is my dad. We are sitting across from each other in my parents' bedroom, him in a rose-colored armchair and me in a desk chair. It's the same room where, decades ago, he calmly forgave me after I confessed that I'd driven the family station wagon through a garage door. Now it's May 2016, he is 80 years old, and I am holding a digital audio recorder. Sensing that I don't quite know how to proceed, my dad hands me a piece of notepaper marked with a skeletal outline in his handwriting. It consists of just a few broad headings: "Family History." "So … do you want to take one of these cat egories and dive into it?" "I want to dive in," he says confidently. "Well, in the first place, my mother was born in the village of Kehries--K-e-h-r-i-e-s--on the Greek island of Evia …" With that, the session is under way. We are sitting here, doing this, because my father has recently been diagnosed with stage IV lung cancer. The disease has metastasized widely throughout his body, including his bones, liver, and brain. It is going to kill him, probably in a matter of months. So now my father is telling the story of his life. This will be the first of more than a dozen sessions, each lasting an hour or more. As my audio recorder runs, he describes how he used to explore caves when he was growing up; how he took a job during college loading ice blocks into railroad boxcars.

Artificial intelligence (AI) has become a buzzword, pushing established software vendors to introduce it into their product strategy, "creating considerable confusion in the process", according to IT researcher Gartner Inc. Gartner Inc, on Tuesday, predicted that by 2020, AI technologies will be virtually pervasive in almost every new software product and service and will be one of the top five investment priorities for more than 30% of CIOs, globally. Gartner defined AI as systems that change behaviors without being explicitly programmed, based on data collected, usage analysis and other observations. The researcher said that in May 2017, the term artificial intelligence was the seventh most popular term on gartner.com, The term did not feature in top 100 search terms back in January 2016. "AI offers exciting possibilities, but unfortunately, most vendors are focused on the goal of simply building and marketing an AI-based product rather than first identifying needs, potential uses and the business value to customers," said Jim Hare, research vice president at Gartner.

For more than 250 years the fundamental drivers of economic growth have been technological innovations. The most important of these are what economists call general-purpose technologies -- a category that includes the steam engine, electricity, and the internal combustion engine. The internal combustion engine, for example, gave rise to cars, trucks, airplanes, chain saws, and lawnmowers, along with big-box retailers, shopping centers, cross-docking warehouses, new supply chains, and, when you think about it, suburbs. Companies as diverse as Walmart, UPS, and Uber found ways to leverage the technology to create profitable new business models. The most important general-purpose technology of our era is artificial intelligence, particularly machine learning (ML) -- that is, the machine's ability to keep improving its performance without humans having to explain exactly how to accomplish all the tasks it's given. Within just the past few years machine learning has become far more effective and widely available. We can now build systems that learn how to perform tasks on their own. Why is this such a big deal? First, we humans know more than we can tell: We can't explain exactly how we're able to do a lot of things -- from recognizing a face to making a smart move in the ancient Asian strategy game of Go. Prior to ML, this inability to articulate our own knowledge meant that we couldn't automate many tasks. Second, ML systems are often excellent learners.

In 1899, the world's most powerful nations signed a treaty at The Hague that banned military use of aircraft, fearing the emerging technology's destructive power. Five years later the moratorium was allowed to expire, and before long aircraft were helping to enable the slaughter of World War I. "Some technologies are so powerful as to be irresistible," says Greg Allen, a fellow at the Center for New American Security, a non-partisan Washington DC think tank. "Militaries around the world have essentially come to the same conclusion with respect to artificial intelligence." Allen is coauthor of a 132-page new report on the effect of artificial intelligence on national security. One of its conclusions is that the impact of technologies such as autonomous robots on war and international relations could rival that of nuclear weapons.