Decision trees are a well-established tool in machine learning for classification and regression tasks. In this paper, we introduce a novel non-parametric Bayesian model that uses variational inference to approximate a posterior distribution over the space of stochastic decision trees. We evaluate the model's performance on 18 datasets and demonstrate its competitiveness with other state-of-the-art methods in regression tasks. We also explore its application to causal inference problems. We provide a fully vectorized implementation of our algorithm in PyTorch.

Decision trees are a well-established tool in machine learning for classification and regression tasks. In this paper, we introduce a novel non-parametric Bayesian model that uses variational inference to approximate a posterior distribution over the space of stochastic decision trees. We evaluate the model's performance on 18 datasets and demonstrate its competitiveness with other state-of-the-art methods in regression tasks. We also explore its application to causal inference problems. We provide a fully vectorized implementation of our algorithm in PyTorch.

Although science and the public celebrated the amazing predictive power of the new machine learningmethods, many researchers are left with some unease, simply because prediction does not imply causation. The ability to uncover causal relations is, however, at the core of most questions concerning the effects of particular policies, medical treatments, marketing campaigns, businessdecisions, etc. (see Athey, 2017, for a recent discussion). The recently rapidly expanding causal machine learning literature holds great promise for the improved estimation of causal effects by merging the statistics and econometrics literature oncausality with the supervised statistical and machine learning (ML) literature focussing on prediction. The classical causality literature clarifies the conditions needed for being able to estimate causal effects. It also shows how to transform a counterfactual causal problem into specific prediction problems (e.g., Imbens and Wooldridge, 2009). The latter literature on ML provides tools that can be highly effective in solving prediction problems (e.g.