TATi-Thermodynamic Analytics ToolkIt: TensorFlow-based software for posterior sampling in machine learning applications

The fundamental role of neural networks (NNs) is readily apparent from their widespread use in machine learning in applications such as natural language processing [72], social network analysis [26], medical diagnosis [6, 35], vision systems [66], and robotic path planning [44]. The greatest success of these models lies in their flexibility, their ability to represent complex, nonlinear relationships in high-dimensional data sets, and the availability of frameworks that allow NNs to be implemented on rapidly evolving GPU platforms [40, 29]. The industrial appetite for deep learning has led to very rapid expansion of the subject in recent years, although, as pointed out by Dunson [19], at times the mathematical and theoretical understanding of these methods has been swept aside in the rush to advance the methodology. The potential impact on society of machine learning algorithms demands that their exposition and use be subject to the highest standards of clarity, ease of interpretation, and uncertainty quantification. Typical NN training seeks to optimize the parameters of the network (biases and weights) under the constraint that the training data set is well approximated [28, 23].