Model Cards for AI Teammates: Comparing Human-AI Team Familiarization Methods for High-Stakes Environments

-- We compare three methods of familiarizing a human with an artificial intelligence (AI) teammate ("agent") prior to operation in a collaborative, fast-paced intelligence, surveillance, and reconnaissance (ISR) environment. In a between-subjects user study (n=60), participants either read documentation about the agent, trained alongside the agent prior to the mission, or were given no familiarization. Results showed that the most valuable information about the agent included details of its decision-making algorithms and its relative strengths and weaknesses compared to the human. This information allowed the familiarization groups to form sophisticated team strategies more quickly than the control group. Documentation-based familiarization led to the fastest adoption of these strategies, but also biased participants towards risk-averse behavior that prevented high scores. Participants familiarized through direct interaction were able to infer much of the same information through observation, and were more willing to take risks and experiment with different control modes, but reported weaker understanding of the agent's internal processes. Significant differences were seen between individual participants' risk tolerance and methods of AI interaction, which should be considered when designing human-AI control interfaces. Based on our findings, we recommend a human-AI team familiarization method that combines AI documentation, structured in-situ training, and exploratory interaction. I. INTRODUCTION Governments have long sought to reduce reliance on human operators in high-stakes domains such as aircraft surveillance, coastal scanning, and mountainous search-and-rescue. Simultaneously, the capabilities of deep learning techniques and accessibility of high-powered compute resources have made autonomous teammates technically viable for many use cases. In recent years governments have begun supporting research to apply embodied artificial intelligence (AI) platforms to reduce the number of humans sent into high-risk scenarios by increasing the level of authority granted to AI systems in human-AI teams.