Laws of thermodynamics for exponential families

Most learning problems can be solved by minimization of log loss. This bare fact is inescapable in modern AI and machine learning - the variety is in the details. What is the space of measured data? What is the support of the distribution? Changing such properties of the problem fundamentally changes learning behavior, leading to the variety of modeling approaches successfully used in data science. But for many inference and decision-making tasks, log loss can be axiomatically inescapable. We explore such loss minimization problems in the language of statistical mechanics, which studies how systems of "particles" like atoms can be approximately described by relatively few bulk properties. There is a direct analogue to modeling, where large datasets are described by relatively few model parameters.