The selection committee for the ACM SIGAI Industry Award for Excellence in Artificial Intelligence (AI) is pleased to announce that the Decision Service created by the Real World Reinforcement Learning Team from Microsoft, has been chosen as the winner of the inaugural 2019 award. The committee was impressed with the identification and development of cutting-edge research on contextual-bandit learning, the manifest cooperation between research and development efforts, the applicability of the decision support throughout the broad range of Microsoft products, and the quality of the final systems. All these aspects made the Microsoft team well worthy of this award. See the call for nominations.
About a year ago I was at a tech conference and there was one topic that threatened to overwhelm all others: no matter how a conversation started, it always ended up being about the fear and uncertainty of what would happen when the robots take over our jobs. Last month I attended O'Reilly's Artificial Intelligence conference in San Francisco and, perhaps not unexpectedly, the dominant topics were completely different.
Machine learning's advances have led to new ideas about the feasibility and importance of machine ethics keeping pace, with increasing emphasis on safety, containment, and alignment. This paper addresses a recent suggestion that inverse reinforcement learning (IRL) could be a means to so-called "value alignment.'' We critically consider how such an approach can engage the social, norm-infused nature of ethical action and outline several features of ethical appraisal that go beyond simple models of behavior, including unavoidably temporal dimensions of norms and counterfactuals. We propose that a hybrid approach for computational architectures still offers the most promising avenue for machines acting in an ethical fashion.
Human beings begin to learn the difference before we learn to speak--and thankfully so. We owe much of our success as a species to our capacity for moral reasoning. It's the glue that holds human social groups together, the key to our fraught but effective ability to cooperate. We are (most believe) the lone moral agents on planet Earth--but this may not last. The day may come soon when we are forced to share this status with a new kind of being, one whose intelligence is of our own design. Robots are coming, that much is sure. They are coming to our streets as self-driving cars, to our military as automated drones, to our homes as elder-care robots--and that's just to name a few on the horizon (Ten million households already enjoy cleaner floors thanks to a relatively dumb little robot called the Roomba). What we don't know is how smart they will eventually become.
Emerging AI systems will be making more and more decisions that impact the lives of humans in a significant way. It is essential, then, that these AI systems make decisions that take into account the desires, goals, and preferences of other people, while simultaneously learning about what those preferences are. In this work, we argue that the reinforcement-learning framework achieves the appropriate generality required to theorize about an idealized ethical artificial agent, and offers the proper foundations for grounding specific questions about ethical learning and decision making that can promote further scientific investigation. We define an idealized formalism for an ethical learner, and conduct experiments on two toy ethical dilemmas, demonstrating the soundness and flexibility of our approach. Lastly, we identify several critical challenges for future advancement in the area that can leverage our proposed framework.