Reinforcement learning (RL) agents use trial and error to learn action policies for environment states. Environments with continuous action spaces are far more challenging for RL than those with discrete actions because there are infinite possible continuous action values from which to choose. Dynamic environments create additional challenges for RL agents, which must adjust rapidly to changes. We recently introduced REINFORCE SUN, a superclass of REINFORCE with Gaussian units, that allows for stochasticity at different levels of granularity in artificial neural networks (synapse, unit, or network), and have shown that moving stochasticity to synapses greatly aids RL in both static and dynamic environments with continuous action spaces. However, we also found that performance in dynamic environments remained substantially lower than desired. To rectify this, we here consider alternative parameter update equations for learning in dynamic environments. These equations form the core of Stochastic Synapse Reinforcement Learning (SSRL), which we here generalize to create S*RL, a superclass of SSRL that allows for stochasticity at these levels. Empirical results using multi-dimensional robot inverse kinematic data sets show that S*RL update equations greatly outperform traditional REINFORCE equations in dynamic, continuous state and action spaces.

The Twenty-Ninth AAAI Conference on Artificial Intelligence, (AAAI-15) was held in January 2015 in Austin, Texas (USA) The conference program was cochaired by Sven Koenig and Blai Bonet. This report contains reflective summaries of the main conference, the robotics program, the AI and robotics workshop, the virtual agent exhibition, the what's hot track, the competition panel, the senior member track, student and outreach activities, the student abstract and poster program, the doctoral consortium, the women's mentoring event, and the demonstrations program.

The AAAI-15 organizing committee of about 60 researchers arranged many of the traditional AAAI events, including the Innovative Applications of Artificial Intelligence (IAAI) Conference, tutorials, workshops, the video competition, senior member summary talks (on well-developed bodies of research or important new research areas), and What's Hot talks (on research trends observed in other AIrelated conferences and, for the first time, competitions). Innovations of AAAI-15 included software and hardware demonstration programs, a virtual agent exhibition, a computer-game showcase, a funding information session with program directors from different funding agencies, and Blue Sky Idea talks (on visions intended to stimulate new directions in AI research) with awards funded by the CRA Computing Community Consortium. Seven invited talks surveyed AI research in academia and industry and its impact on society. Attendees kept track of the program through a smartphone app as well as social media channels.

Hail causes billions of dollars in losses by damaging buildings, vehicles, and crops. Improving the spatial and temporal accuracy of hail forecasts would allow people to mitigate hail damage. We have developed an approach to forecasting hail that identifies potential hail storms in storm-scale numerical weather prediction models and matches them with observed hailstorms. Machine learning models, including random forests, gradient boosting trees, and linear regression, are used to predict the expected hail size from each forecast storm. The individual hail size forecasts are merged with a spatial neighborhood ensemble probability technique to produce a consensus probability of hail at least 25.4 mm in diameter. The system was evaluated during the 2014 National Oceanic and Atmospheric Administration Hazardous Weather Testbed Experimental Forecast Program and compared with a physics-based hail size model. The machine-learning-based technique shows advantages in producing smaller size errors and more reliable probability forecasts. The machine learning approaches correctly predicted the location and extent of a significant hail event in eastern Nebraska and a marginal severe hail event in Colorado.

Infants at risk for cerebral palsy are at a severe disadvantage in learning to crawl as compared with typically developing infants. An assistive system is being created at the University of Oklahoma to improve these children's crawling abilities. The infants are: outfitted with a suit that allows kinematic reconstruction of their movements; EEG monitoring of their neural responses; and placed in an assistive robot that can amplify the effectiveness of their crawling actions and reduce the required weight bearing for successful prone locomotion. The system can also map their attempted motions into a library of recognized movements, and create directed robot motion even when the subject has not generated any propulsive forces on their own.

We introduce a Java graphical gaming framework that enables students in an introductory artificial intelligence (AI) course to immediately apply and visualize the topics from class. We have used this framework in teaching a mixed undergraduate/graduate AI course for six years. We believe that the use of games motivates students. The graphical nature of each game enables students to quickly see how well their algorithm works. Because the topics in an introductory AI course vary widely, students apply their algorithms to multiple game environments. A final challenging environment enables them to tie together the concepts for the entire semester.

As educators, we have high hopes for Electronic Tangibles (ETs), we expect ETs to: Interest more students in the study of computing Broaden students' views of computing Invite non-majors to learn something about the computing Attract students to computer science as a major Help students learn about particular ETs Attract students to our classes by incorporating a flashy ET in the course material Improve student understanding of some difficult topics Maintain student interest throughout the class However some important questions arise: Can we and should we extend these benefits throughout the K-20 curriculum? And if we can't, are we guilty of bait-and-switch?