There has been a surge of recent interest in automatically learning policies to target treatment decisions based on rich individual covariates. A common approach is to train a machine learning model to predict counterfactual outcomes, and then select the policy that optimizes the predicted objective value. In addition, practitioners also want confidence that the learned policy has better performance than the incumbent policy according to downstream policy evaluation. However, due to the winner's curse-an issue where the policy optimization procedure exploits prediction errors rather than finding actual improvements-predicted performance improvements are often not substantiated by downstream policy optimization. To address this challenge, we propose a novel strategy called inference-aware policy optimization, which modifies policy optimization to account for how the policy will be evaluated downstream. Specifically, it optimizes not only for the estimated objective value, but also for the chances that the policy will be statistically significantly better than the observational policy used to collect data. We mathematically characterize the Pareto frontier of policies according to the tradeoff of these two goals. Based on our characterization, we design a policy optimization algorithm that uses machine learning to predict counterfactual outcomes, and then plugs in these predictions to estimate the Pareto frontier; then, the decision-maker can select the policy that optimizes their desired tradeoff, after which policy evaluation can be performed on the test set as usual. Finally, we perform simulations to illustrate the effectiveness of our methodology.

Workers spend a significant amount of time learning how to make good decisions. Evaluating the efficacy of a given decision, however, can be complicated -- e.g., decision outcomes are often long-term and relate to the original decision in complex ways. Surprisingly, even though learning good decision-making strategies is difficult, they can often be expressed in simple and concise forms. Focusing on sequential decision-making, we design a novel machine learning algorithm that is capable of extracting "best practices" from trace data and conveying its insights to humans in the form of interpretable "tips". Our algorithm selects the tip that best bridges the gap between the actions taken by human workers and those taken by the optimal policy in a way that accounts for which actions are consequential for achieving higher performance. We evaluate our approach through a series of randomized controlled experiments where participants manage a virtual kitchen. Our experiments show that the tips generated by our algorithm can significantly improve human performance relative to intuitive baselines. In addition, we discuss a number of empirical insights that can help inform the design of algorithms intended for human-AI interfaces. For instance, we find evidence that participants do not simply blindly follow our tips; instead, they combine them with their own experience to discover additional strategies for improving performance.

We study the problem of transfer learning, observing that previous efforts to understand its information-theoretic limits do not fully exploit the geometric structure of the source and target domains. In contrast, our study first illustrates the benefits of incorporating a natural geometric structure within a linear regression model, which corresponds to the generalized eigenvalue problem formed by the Gram matrices of both domains. We next establish a finite-sample minimax lower bound, propose a refined model interpolation estimator that enjoys a matching upper bound, and then extend our framework to multiple source domains and generalized linear models. Surprisingly, as long as information is available on the distance between the source and target parameters, negative-transfer does not occur. Simulation studies show that our proposed interpolation estimator outperforms state-of-the-art transfer learning methods in both moderate- and high-dimensional settings.

A new machine learning approach to COVID-19 testing has produced encouraging results in Greece. The technology, named Eva, dynamically used recent testing results collected at the Greek border to detect and limit the importation of asymptomatic COVID-19 cases among arriving international passengers between August and November 2020, which helped contain the number of cases and deaths in the country. The findings of the project are explained in a paper titled "Deploying an Artificial Intelligence System for COVID-19 Testing at the Greek Border," authored by Hamsa Bastani, a Wharton professor of operations, information and decisions and affiliated faculty at Analytics at Wharton; Kimon Drakopoulos and Vishal Gupta from the University of Southern California; Jon Vlachogiannis from investment advisory firm Agent Risk; Christos Hadjicristodoulou from the University of Thessaly; and Pagona Lagiou, Gkikas Magiorkinis, Dimitrios Paraskevis and Sotirios Tsiodras from the University of Athens. The analysis showed that Eva on average identified 1.85 times more asymptomatic, infected travelers than what conventional, random surveillance testing would have achieved. During the peak travel season of August and September, the detection of infection rates was up to two to four times higher than random testing.

A new machine learning approach to COVID-19 testing has produced encouraging results in Greece. The technology, named Eva, dynamically used recent testing results collected at the Greek border to detect and limit the importation of asymptomatic COVID-19 cases among arriving international passengers between August and November 2020, which helped contain the number of cases and deaths in the country. The findings of the project are explained in a paper titled "Deploying an Artificial Intelligence System for COVID-19 Testing at the Greek Border," authored by Hamsa Bastani, a Wharton professor of operations, information and decisions and affiliated faculty at Analytics at Wharton; Kimon Drakopoulos and Vishal Gupta from the University of Southern California; Jon Vlachogiannis from investment advisory firm Agent Risk; Christos Hadjicristodoulou from the University of Thessaly; and Pagona Lagiou, Gkikas Magiorkinis, Dimitrios Paraskevis and Sotirios Tsiodras from the University of Athens. The analysis showed that Eva on average identified 1.85 times more asymptomatic, infected travelers than what conventional, random surveillance testing would have achieved. During the peak travel season of August and September, the detection of infection rates was up to two to four times higher than random testing.

A key challenge for deploying deep neural networks (DNNs) in safety critical settings is the need to provide rigorous ways to quantify their uncertainty. In this paper, we propose a novel algorithm for constructing predicted classification confidences for DNNs that comes with provable correctness guarantees. Our approach uses Clopper-Pearson confidence intervals for the Binomial distribution in conjunction with the histogram binning approach to calibrated prediction. In addition, we demonstrate how our predicted confidences can be used to enable downstream guarantees in two settings: (i) fast DNN inference, where we demonstrate how to compose a fast but inaccurate DNN with an accurate but slow DNN in a rigorous way to improve performance without sacrificing accuracy, and (ii) safe planning, where we guarantee safety when using a DNN to predict whether a given action is safe based on visual observations. In our experiments, we demonstrate that our approach can be used to provide guarantees for state-of-the-art DNNs. Due to the recent success of machine learning, there has been increasing interest in using predictive models such as deep neural networks (DNNs) in safety-critical settings, such as robotics (e.g., obstacle detection (Ren et al., 2015) and forecasting (Kitani et al., 2012)) and healthcare (e.g., diagnosis (Gulshan et al., 2016; Esteva et al., 2017) and patient care management (Liao et al., 2020)).

Google Maps is a triumph of artificial intelligence in action, with the ability to guide us from one place to another using some impressive machine learning technology. But while the routing part of Google Maps doesn't need too many humans in the mix, manually tracing the roads on the aerial images to make them machine usable is incredibly time-consuming and mundane. As a result, even with thousands of hours spent on this task, Google employees still haven't managed to map the majority of the 20 million-plus miles of roadways that stretch around the world. Fortunately, researchers from the Massachusetts Institute of Technology's Computer Science and Artificial Intelligence Laboratory (CSAIL) may have come up with a solution. They developed an automated method to build roadmaps which is 45 percent more accurate than existing methods.

Gaps in maps are a problem, particularly for systems being developed for self-driving cars. To address the issue, researchers from MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) have created RoadTracer, an automated method to build road maps that's 45 percent more accurate than existing approaches. Using data from aerial images, the team says that RoadTracer is not just more accurate, but more cost-effective than current approaches. MIT professor Mohammad Alizadeh says that this work will be useful both for tech giants like Google and for smaller organizations without the resources to curate and correct large amounts of errors in maps. "RoadTracer is well-suited to map areas of the world where maps are frequently out of date, which includes both places with lower population and areas where there's frequent construction," says Alizadeh, one of the co-authors of a new paper about the system.

As someone who watches technology trends closely as part of my business, I have been thinking about the future impact of all the technology innovations and automation we are currently experiencing and on the cusp of achieving. Many of the headlines I read about these trends -- and even some I write -- predict some pretty negative consequences right along with the monumental achievements and improvements. While improvements in machine learning, artificial intelligence, big data, and robot automation could mean huge advances in medicine, science, commerce and human understanding, it's also undeniable that there will be consequences as well. These technological advances represent a significant challenge to capitalism. Together, they are poised to potentially create jobless growth and the paradox of an exponentially growing number of products, manufactured more and more efficiently, but with rising unemployment and underemployment, falling real wages and stagnant living standards.