While this reality has become more tangible in recent years through consumer technology, such as Amazon's Alexa or Apple's Siri, the applications of AI software are already widespread, ranging from credit card fraud detection at VISA to payload scheduling operations at NASA to insider trading surveillance on the NASDAQ. Broadly defined as the imitation of human cognition by a machine, recent interest in AI has been driven by advances in machine learning, in which computer algorithms learn from data without human direction.1 Most sophisticated processes that involve some form of prediction generated from a large data set use this type of AI, including image recognition, web-search, speech-to-text language processing, and e-commerce product recommendations.2 AI is increasingly incorporated into devices that consumers keep with them at all times, such as smartphones, and powers consumer technologies on the horizon, such as self-driving cars. And there is anticipation that these advances will continue to accelerate: a recent survey of leading AI researchers predicted that, within the next 10 years, AI will outperform humans in transcribing speech, translating languages, and driving a truck.3
Finding the optimal signal timing strategy is a difficult task for the problem of large-scale traffic signal control (TSC). Multi-Agent Reinforcement Learning (MARL) is a promising method to solve this problem. However, there is still room for improvement in extending to large-scale problems and modeling the behaviors of other agents for each individual agent. In this paper, a new MARL, called Cooperative double Q-learning (Co-DQL), is proposed, which has several prominent features. It uses a highly scalable independent double Q-learning method based on double estimators and the UCB policy, which can eliminate the over-estimation problem existing in traditional independent Q-learning while ensuring exploration. It uses mean field approximation to model the interaction among agents, thereby making agents learn a better cooperative strategy. In order to improve the stability and robustness of the learning process, we introduce a new reward allocation mechanism and a local state sharing method. In addition, we analyze the convergence properties of the proposed algorithm. Co-DQL is applied on TSC and tested on a multi-traffic signal simulator. According to the results obtained on several traffic scenarios, Co- DQL outperforms several state-of-the-art decentralized MARL algorithms. It can effectively shorten the average waiting time of the vehicles in the whole road system.
The terms "machine learning" and "artificial intelligence" (AI) conjure up feelings that are equal parts fear and fascination. Until recently, the prospect of a piece of software making human-like decisions resided safely in the far-fetched expectations of 1960s-era computer scientists or the plot lines of science fiction novels. Today, however, after decades of unmet expectations, we finally have AI systems that are beginning to influence our lives in tangible ways. Voice recognition systems like Amazon's Echo and Apple's Siri, and once-unimaginable fantasies like self-driving cars, are on the market for consumers, with more exciting life-like systems to come. We have also seen a few early signs of robotic autonomy that makes us feel uneasy, like the Russian robot that learned how to escape the lab!
Apple was the first to introduce a digital assistant when it acquired Siri and baked it into the OS. The makers of Siri went on to create Viv, a realisation of their original vision for Siri. Viv was acquired by Samsung this year. Despite the initial head start with Siri, Apple lost ground over the subsequent years by being very secretive about its research. As Apple employees were not allowed to publish research, the best talent was not attracted to the company.
Morris, Robert (NASA Ames Research Center) | Chang, Mai Lee (Johnson Space Center) | Archer, Ronald (Lockheed Martin) | Cross, Ernest V (Lockheed Martin) | Thompson, Shelby (Lockheed Martin) | Franke, Jerry (Lockheed Martin) | Garrett, Robert (Lockheed Martin) | Malik, Waqar (University of California-Santa Cruz Affiliated Research Center) | McGuire, Kerry (NASA Johnson Space Center) | Hemann, Garrett (Carnegie Mellon University)
We introduce an application of self-driving vehicle technology to the problem of towing aircraft at busy airports from gate to runway and runway to gate. Autonomous towing can be supervised by human ramp- or ATC controllers, pilots, or ground crew. The controllers provide route information to the tugs, assisted by an automated route planning system. The planning system and tower and ground controllers work in conjunction with the tugs to make tactical decisions during operations to ensure safe and effective taxiing in a highly dynamic environment. We argue here for the potential for significantly reducing fuel emissions, fuel costs, and community noise, while addressing the added complexity of air terminal operations by increasing efficiency and reducing human workload. This paper describes work-in-progress for developing concepts and capabilities for autonomous engines-off taxiing using towing vehicles.