How to Increase Computational Efficiency for PReLU in CUDA -- OneFlow Performance Optimization

PReLU is an activation function that is frequently used in InsightFace. It has two operating modes: PReLU(1) and PReLU(channels). For the latter, PReLU is equivalent to a binary broadcast operation. In this article, we are going to talk about optimizing the broadcast operations in CUDA. PReLU is an activation function that is frequently used in InsightFace. InsightFace adopts the second mode of PReLU.