Early Impacts of M365 Copilot

Dillon, Eleanor Wiske, Jaffe, Sonia, Peng, Sida, Cambon, Alexia

arXiv.org Artificial Intelligence 

New generative AI tools have been developing rapidly and are now widely used, including by workers in doing their jobs. Microsoft worked with firms across industries to run a large field experiment to measure how access to one of these tools changes work patterns. The experiment ran during the early rollout of Microsoft's M365 Copilot (Copilot), a multi-part generative AI tool that integrates generative AI into components of Microsoft's M365 suite (including Word, PowerPoint, Outlook, and Teams). M365 Copilot is designed as a general purpose tool to help workers digest information by summarizing emails, meetings, or documents, create new content by drafting emails, documents, or slide decks, and retrieve information either from the web or across any company email, chat, or document to which the worker has access. We worked with firms to randomize access to Copilot and got permission to use several months of anonymized metadata on workers' behaviors in Outlook, Teams, and Office, both before and after access to Copilot.