Hidden Activations Are Not Enough: A General Approach to Neural Network Predictions