Explainable Machine Learning in Deployment

Explainable machine learning seeks to provide various stak ehold-ers withinsights into modelbehavior via feature importancescores, counterfactual explanations, and influential samples, among other techniques. Recent advances in this line of work, however, h ave gone without surveys of how organizations are using these te ch-niques in practice. This study explores how organizations v iew and use explainability for stakeholder consumption. We find that the majority of deployments are not for end users affected by t he model but for machine learning engineers, who use explainability to debug the model itself. There is a gap between explainability in practice and the goal of publictransparency, since explanations primarily serve internal stakeholders rather than external on es. Our study synthesizes the limitations with current explainabi lity techniques that hamper their use for end users. To facilitate end user interaction, we develop a framework for establishing clear goals for explainability, including a focus on normative desiderata.