Combining Gradients and Probabilities for Heterogeneous Approximation of Neural Networks