Appendix . StochasticAdaptiveActivationFunction

ASH activation function inthe early layer exhibits a small threshold (large percentile) to retain substantial information, whereas ASH in deeper layers exhibits a small comparative percentile to rectify futile information. Supplementary Figure 1 illustrates the training graph of loss values and validation accuracies. Supplementary Figure 6: Samples generated byDCGAN withReLU, Swish, andASHactivation functionsusingcelebAdataset. Supplementary Figure 1 illustrates the generated samples by DCGAN (Radford et al., 2015) with ReLU, Swish, andASH activation functions using celebA dataset (Yangetal.,2015). ASH activation function that rectified top-k% percentile could be modified into various versions.