Transparent Classification with Multilayer Logical Perceptrons and Random Binarization

Models with transparent inner structure and high classification performance are required to reduce potential risk and provide trust for users in domains like health care, finance, security, etc. However, existing models are hard to simultaneously satisfy the above two properties. In this paper, we propose a new hierarchical rule-based model for classification tasks, named Concept Rule Sets (CRS), which has both a strong expressive ability and a transparent inner structure. To address the challenge of efficiently learning the non-differentiable CRS model, we propose a novel neural network architecture, Multilayer Logical Perceptron (MLLP), which is a continuous version of CRS. Using MLLP and the Random Binarization (RB) method we proposed, we can search the discrete solution of CRS in continuous space using gradient descent and ensure the discrete CRS acts almost the same as the corresponding continuous MLLP . Experiments on 12 public data sets show that CRS outperforms the state-of-the-art approaches and the complexity of the learned CRS is close to the simple decision tree. Introduction Relying on strong ability of data modeling, machine learning, especially deep learning, becomes the main paradigm for decision-making systems (Goodfellow et al. 2016; Doshi-V elez and Kim 2017). The decision-making systems have widespread usage in important areas such as medicine, finance, politics, as well as law, where people need the explanations why decisions are made to ensure their safety and protect their rights (Goodman and Flaxman 2016; Lipton 2016). As a result, the demand for the transparency of machine learning methods is increasing, which is crucial for earning the trust of users (Doshi-V elez and Kim 2017) and reducing potential risks and bugs (Chu et al. 2018). However, most of the machine learning models can hardly ensure good predictive ability and transparency at the same time, and sacrificing transparency for good performance could result in serious consequences.