Gradients are Not All You Need