Appendix . Stochastic Adaptive Activation Function

Intuitively, ASH activation function is the threshold-based activation function rectifying inputs, and we obtained the following properties: Property 1. ASH activation function is parametric. ASH activation function in the early layer exhibits a small threshold (large percentile) to retain substantial information, whereas ASH in deeper layers exhibits a small comparative percentile to rectify futile information. Property 2. ASH activation function provides output concerning the contexts of the input. Supplementary Figure 1 illustrates the training graph of loss values and validation accuracies. In addition, the y-axis indicates the range of (0, 0.8). 3 Appendix D. Classification task Supplementary Figure 1 illustrates the GRAD-CAM (Selvaraju et al., 2017) samples by using ResNet-164 and Dense-Net models with ReLU, Swish, and ASH activation function in the classification task In Supplementary Figure 1 Property 1 is clearly illustrated.