Uncovering the Power and Limitations of the TanH Activation Function in Neural Networks