Beyond One-Hot-Encoding: Injecting Semantics to Drive Image Classifiers

Deep Learning (DL) models have become the go-to method for addressing numerous Computer Vision (CV) tasks, such as image classification. Unlike traditional approaches that require manual feature extraction, DL streamlines the development of end-to-end pipelines that seamlessly integrate images as inputs to the learning process, thereby automating feature extraction and enhancing overall efficiency. This automation enables the training of DL models over extensive image datasets, which subsequently leads to enhanced model accuracy. However, the "black-box" nature of DL models presents challenges, as Machine Learning (ML) practitioners often struggle to understand the chain of transformations that a DL model adopts to map an image into the final prediction. This lack of transparency is considered to be hampering the adoption of DL models in real-world scenarios, due to a plethora of reasons: lack of trust from domain experts, impossibility of thorough debugging from practitioners, lack of compliance to legal requirements regarding explainability, and potential systemic bias in the trained model [18]. The research field of eXplainable Artificial Intelligence (XAI) tackles this problem by trying to provide more insights about the inner decision process of ML models [11]. However, most XAI techniques for CV are post-hoc: they are applied on trained ML models, and typically try to correlated portions of the image to the resulting label by means of input perturbation or maskings [29, 31, 33]. A few other approaches try to modify the training procedure itself, hoping to gain more control over the model's internals, while at the same time maintaining competitive classification performances. With this in mind, we remark how the standard DL pipeline for image classification trains the model to learn a mapping from images to labels.