The Myth of Model Interpretability

Update: I have since refined these ideas in The Mythos of Model Interpretability, an academic paper presented at the 2016 ICML Workshop on Human Interpretability of Machine Learning. Neural networks, on the other hand, are black boxes. By this, it's suggested that we can pass input in, and observe what comes out, but we lack the ability to reason about what happened in the middle. To confirm the prevalence of this narrative, I ran a Google search for "neural network black box", yielding 2,410,000 results. By comparison, "logistic regression black box" turns up 600,000 results.