Artificial intelligence (AI) has received significant attention in recent years, primarily due to breakthroughs in game playing, computer vision, and natural language processing that captured the imagination of the scientific community and the public at large. Many businesses, industries, and academic disciplines are now contemplating the application of AI to their own challenges. The federal government in the US and other countries have also invested significantly in advancing AI research and created funding initiatives and programs to promote greater collaboration across multiple communities. Some of the investment examples in the US include the establishment of the National AI Initiative Office, the launch of the National AI Research Resource Task Force, and more recently, the establishment of the National AI Advisory Committee. In 2021 INFORMS and ACM SIGAI joined together with the Computing Community Consortium (CCC) to organize a series of three workshops. The objective for this workshop series is to explore ways to exploit the synergies of the AI and Operations Research (OR) communities to transform decision making.

Through a series of federal initiatives and orders, the U.S. Government has been making a concerted effort to ensure American leadership in AI. These broad strategy documents have influenced organizations such as the United States Department of the Air Force (DAF). The DAF-MIT AI Accelerator is an initiative between the DAF and MIT to bridge the gap between AI researchers and DAF mission requirements. Several projects supported by the DAF-MIT AI Accelerator are developing public challenge problems that address numerous Federal AI research priorities. These challenges target priorities by making large, AI-ready datasets publicly available, incentivizing open-source solutions, and creating a demand signal for dual use technologies that can stimulate further research. In this article, we describe these public challenges being developed and how their application contributes to scientific advances.

In recent years there have been great strides in artificial intelligence (AI), with games often serving as challenge problems, benchmarks, and milestones for progress. Poker has served for decades as such a challenge problem. Past successes in such benchmarks, including poker, have been limited to two-player games. However, poker in particular is traditionally played with more than two players. Multiplayer games present fundamental additional issues beyond those in two-player games, and multiplayer poker is a recognized AI milestone.

In this demo paper, we introduce the DARPA D3M program for automatic machine learning (ML) and JPL's MARVIN tool that provides an environment to locate, annotate, and execute machine learning primitives for use in ML pipelines. MARVIN is a web-based application and associated back-end interface written in Python that enables composition of ML pipelines from hundreds of primitives from the world of Scikit-Learn, Keras, DL4J and other widely used libraries. MARVIN allows for the creation of Docker containers that run on Kubernetes clusters within DARPA to provide an execution environment for automated machine learning. MARVIN currently contains over 400 datasets and challenge problems from a wide array of ML domains including routine classification and regression to advanced video/image classification and remote sensing.

WE NEED BETTER STANDARDS FOR AI RESEARCH The state of the art in any science includes the criteria for evaluating research. Like every other aspect of the science, it has to be developed. In my previous message I grumbled about there being insufficient basic research, but one of the reasons for this is the difficulty of evaluating whether a piece of research has made basic progress. It seems that evaluation should be based on the kind of advance the research purports to be. I haven't been able to develop a complete set of criteria, but here are some considerations.

Now completing its first year, the High-Performance Knowledge Bases Project promotes technology for developing very large, flexible, and reusable knowledge bases. The project is supported by the Defense Advanced Research Projects Agency and includes more than 15 contractors in universities, research laboratories, and companies. Programs lack knowledge about the world sufficient to understand and adjust to new situations as people do. Consequently, programs have been poor at interpreting and reasoning about novel and changing events, such as international crises and battlefield situations. These problems are more open ended than chess.

Recent years have seen significant technical progress on robot planning, enabling robots to compute actions and motions to accomplish challenging tasks involving driving, flying, walking, or manipulating objects. However, robots that have been commercially deployed in the real world typically have no or minimal planning capability. These robots are often manually programmed, teleoperated, or programmed to follow simple rules. Although these robots are highly successful in their respective niches, a lack of planning capabilities limits the range of tasks for which currently deployed robots can be used. In this article, we highlight key conclusions from a workshop sponsored by the National Science Foundation in October 2013 that summarize opportunities and key challenges in robot planning and include challenge problems identified in the workshop that can help guide future research toward making robot planning more deployable in the real world.

This special issue of AI Magazine brings seven articles presenting extended versions of papers from IAAI 2013. These articles were selected for their description of AI technologies that are either in practical use or close to it. Five of the articles describe deployed application case studies. These articles present fielded AI applications that distinguish themselves for their innovative use of AI technology. One article describes an emerging application.

If it is true that good problems produce good science, then it will be worthwhile to identify good problems, and even more worthwhile to discover the attributes that make them good problems. This discovery process is necessarily empirical, so we examine several challenge problems, beginning with Turing's famous test, and more than a dozen attributes that challenge problems might have. We are led to a contrast between research strategies--the successful "divide and conquer" strategy and the promising but largely untested "developmental" strategy--and we conclude that good challenge problems encourage the latter strategy. More than fifty years ago, Alan Turing proposed a clever test of the proposition that machines can think (Turing 1950). He wanted the proposition to be an empirical, one and he particularly wanted to avoid haggling over what it means for anything to think.

We describe how computational POL modeling integrates diverse artificial intelligence research areas and provides interesting challenges in multiple fields. Simultaneously, these patterns of life impose structure on individual decisions. For example, a pattern of rush hour traffic arises from drivers' decisions to commute at a certain time. Knowledge of rush hour influences individuals' departure times. Modeling POL is not only an academic pursuit.