But that's not what happened. Flash forward 18 months and the end of the process couldn't be more dissimilar from its start. The nervous energy had been replaced with exhaustion. Our optimism that we'd be able to provide an outline for the ways that the New York City government should be using automated decision systems gave way to a fatalistic belief that we may not be able to tackle a problem this big after all. The people in this room were going to decide what role AI should play and what safeguards we should have."

We present an interactive system enabling users to manipulate images to explore the robustness and sensitivity of deep learning image classifiers. Using modern web technologies to run in-browser inference, users can remove image features using inpainting algorithms and obtain new classifications in real time, which allows them to ask a variety of "what if" questions by experimentally modifying images and seeing how the model reacts. Our system allows users to compare and contrast what image regions humans and machine learning models use for classification, revealing a wide range of surprising results ranging from spectacular failures (e.g., a "water bottle" image becomes a "concert" when removing a person) to impressive resilience (e.g., a "baseball player" image remains correctly classified even without a glove or base). We demonstrate our system at The 2018 Conference on Computer Vision and Pattern Recognition (CVPR) for the audience to try it live. Our system is open-sourced at https://github.com/poloclub/interactive-classification. A video demo is available at https://youtu.be/llub5GcOF6w.