This paper surveys the field of reinforcement learning from a computer-science perspective. It is written to be accessible to researchers familiar with machine learning. Both the historical basis of the field and a broad selection of current work are summarized. Reinforcement learning is the problem faced by an agent that learns behavior through trial-and-error interactions with a dynamic environment. The work described here has a resemblance to work in psychology, but differs considerably in the details and in the use of the word reinforcement.''

Unlike many so-called'agent toolkits', like PRS/Jack, Mozart, Alice, and several more, that are aimed mainly at development of systems involving large numbers of highly distributed fairly homogeneous relatively'small' agents, SimAgent can be used for such purposes (and was used in that way for a while by Matthias Scheutz at Notre Dame University) but (like ACT-R, COGENT, and the original SOAR) SimAgent is primarily designed to support design and implementation of very complex agents, each composed of very different interacting components (like a human mind) where the whole thing is embedded in an environment that could be a mixture of physical objects and other agents of many sorts, as half-jokingly depicted here: That schema accommodates a wide variety of specific architecture types, which differ according to which mechanisms and information structures occur in which boxes, and how they are connected to one another and to the environment, as described in this overview. The above diagram is misleading in various ways because it suggest that the perception mechanisms and action mechanisms are separate from each other and can only communicate via the'central' mechanisms, whereas it is clear (as James Gibson pointed out in his 1966 book The Senses Considered as Perceptual Systems) action and perception are deeply integrated, e.g. the constant use of saccades, changes of vergence, changes of focus in vision, and the need to move your hand when it is used to perceive shape, texture, weight, flexibility, hardness, etc. of objects. So a more accurate, but less clear depiction of the ideas in the CogAff architecture schema is the following (with thanks to Dean Petters, for help with this diagram, indicating that action and perception mechanisms overlap, as pointed out by J.J.Gibson in 1966(Referenced above). Revised, more realistic CogAff Architecture Schema, e.g. with deeper integration between action and perception The horizontal discs represent (usually "fuzzy" boundaries between different levels of functionality. It is possible for some of the information-processing mechanisms to straddle two or more layers.

Recently there has been renewed interest in game theory in several research disciplines, with its uses ranging from the modeling of evolution to the design of distributed protocols. In the AI community, game theory is emerging as the dominant formalism for studying strategic and cooperative interaction in multi-agent systems. Classical work provides rich mathematical foundations and equilibrium concepts, but relatively little in the way of computational and representational insights that would allow game theory to scale up to large, complex systems. The rapidly emerging field of computational game theory is addressing such algorithmic issues, and this tutorial will provide a survey of developments so far. As the NIPS community is well-poised to make significant contributions to this area, special emphasis will be placed on connections to more familiar topics.

The Annual Conference on Innovative Applications of Artificial Intelligence offers case studies of deployed applications with measurable benefits whose value depends on the use of AI technology. In addition, many conferences supplement these case studies with papers and invited talksthat address emerging areas of AI technology or applications. IAAI was an independent conference during its early years. Recently, it has been organized as an independent program within the National Conference, with schedules coordinated to allow attendees to move freely between IAAI and National Conference sessions. Most years IAAI and the National Conference papers are published in a joint proceedings.

Today, the National Institutes of Health (NIH) announced the first round of Brain Research through Advancing Innovative Neurotechnologies (BRAIN) Initiative award recipients, including several MIT interdisciplinary teams. The BRAIN Initiative, spearheaded by President Obama in April 2013, challenges the nation's leading scientists to advance our sophisticated understanding of the human mind and discover new ways to treat, prevent, and cure neurological disorders like Alzheimer's, schizophrenia, autism, and traumatic brain injury. "The human brain is one of the most complicated structures in the known universe," said NIH Director Francis Collins. "We have an unprecedented opportunity to develop new technologies that will allow us to map the circuits of the brain, measure activity within those circuits, and understand how their interactions maintain health and modulate human behavior." With the NIH Brain Initiative, the scientific community is charged with accelerating the invention of cutting-edge technologies that can produce dynamic images of complex neural circuits and illuminate the interaction of lightning-fast brain cells.

Researchers in Indiana University's Knowledge Acquisition and Projection Lab -- part of Pervasive Technology Labs -- along with computer scientists from the IU School of Informatics, have completed a project for the U.S. Navy in which they developed key components of the Navy's maintenance Knowledge Projection System. This project, a three-year joint undertaking with Crane Naval Surface Warfare Division and Purdue University, was aimed at developing next-generation diagnostics and maintenance capabilities for shipboard systems. "The need for tele-maintenance and distance support technologies for today's battleships, aircraft carriers and submarines is compelling," said Donald F. (Rick) McMullen, director of the Knowledge Acquisition and Projection Lab (KAPLab). McMullen, along with IU Professor of Computer Science David Leake, served as principal investigator for the KPS project. "The current generation of naval vessels is more complex than ever, and correcting problems with shipboard systems is frequently a team effort involving both ship- and shore-based personnel. In the current environment, distance support is critical to maintaining operational preparedness," McMullen said.

In recent years, the range of sensing technologies has expanded rapidly, whereas sensor devices have become cheaper. This has led to a rapid expansion in condition monitoring of systems, structures, vehicles, and machinery using sensors. Key factors are the recent advances in networking technologies such as wireless communication and mobile ad hoc networking coupled with the technology to integrat... View full abstract» This paper provides a review of the literature in on-road vision-based vehicle detection, tracking, and behavior understanding. Over the past decade, vision-based surround perception has progressed from its infancy into maturity. We provide a survey of recent works in the literature, placing vision-based vehicle detection in the context of sensor-based on-road surround analysis.

Using humanoid robots to study human behavior.

Both generative adversarial networks (GAN) in unsupervised learning and actor-critic methods in reinforcement learning (RL) have gained a reputation for being difficult to optimize. Practitioners in both fields have amassed a large number of strategies to mitigate these instabilities and improve training. Here we show that GANs can be viewed as actor-critic methods in an environment where the actor cannot affect the reward. We review the strategies for stabilizing training for each class of models, both those that generalize between the two and those that are particular to that model. We also review a number of extensions to GANs and RL algorithms with even more complicated information flow. We hope that by highlighting this formal connection we will encourage both GAN and RL communities to develop general, scalable, and stable algorithms for multilevel optimization with deep networks, and to draw inspiration across communities.

Artificial intelligence has wide-reaching implications across all aspects of the business, and now some leaders in the field advocate for this technology to be represented in the C-suite by a chief artificial intelligence officer (CAIO). Andrew Ng, a renowned A.I. researcher and thought leader, recently made that argument in Harvard Business Review article titled "Hiring Your First Chief A.I. Officer." Summarizing his case, Ng wrote, "The benefit of a chief A.I. officer is having someone who can make sure A.I. gets applied across silos." Since every aspect of a business involves the collection and use of data for competitive advantage, a CAIO could look across an organization and assess how different business units can work together to create new competitive advantages. Matthew Buskell, head of sales and business development at Rainbird Technologies, makes a similar case about the cross-functional impact of A.I. in a post on the Rainbird blog titled "Should There be a Chief Artificial Intelligence Officer?"