GRADIEND: Monosemantic Feature Learning within Neural Networks Applied to Gender Debiasing of Transformer Models

We hypothesize that these gradients AI systems frequently exhibit and amplify social biases, contain valuable information for identifying and modifying including gender bias, leading to harmful consequences gender-specific features. Our method aims to learn a in critical areas. This study introduces a novel encoderdecoder feature neuron that encodes gender information from the approach that leverages model gradients to input, i.e., model gradients. Unlike existing approaches learn a single monosemantic feature neuron encoding for extracting monosemantic features (e.g., Bricken et al. gender information. We show that our method can (2023)), our approach enables the learning of a feature neuron be used to debias transformer-based language models, with a desired, interpretable meaning, such as gender.