Is it possible to scale the activation function instead of batch-normalization? • r/MachineLearning

The purpose of using batch-normalization is to keep the distribution of the vectors in a range where the ReLU is non-linear controlled automatically by the Beta and Gamma parameters (which are learnable). I am wondering if the same effect can be achieved by using scaling values for the activation function. Precisely, by multiplying the scaling values to the input of the activation non-linearity, we can stretch and squeeze it in the horizontal direction and by multiplying those values after the activation function, the same can be controlled in the vertical direction. Is there some prior work done on this concept that I can refer to? What are the subtleties involved in doing this compared to the traditional bn- relu non-linearity?