The Principle of Uncertain Maximum Entropy

To whom correspondence should be addressed; E-mail: kbogert@unca.edu. Its resultant solutions have served as a catalyst, facilitating researchers in mapping their empirical observations to the acquisition of unbiased models, whilst deepening the understanding of complex systems and phenomena. However, when we consider situations in which the model elements are not directly observable, such as when noise or ocular occlusion is present, possibilities arise for which standard maximum entropy approaches may fail, as they are unable to match feature constraints. Here we show the Principle of Uncertain Maximum Entropy as a method that both encodes all available information in spite of arbitrarily noisy observations while surpassing the accuracy of some ad-hoc methods. Additionally, we utilize the output of a black-box machine learning model as input into an uncertain maximum entropy model, resulting in a novel approach for scenarios where the observation function is unavailable. We anticipate our principle finding broad applications in diverse fields due to generalizing the traditional maximum entropy method with the ability to utilize uncertain observations. Throughout the sciences, we often desire to estimate some distribution given a number of samples taken from it. This distribution may represent the outcome of a process in the real world whose parameters we are interested in learning. The principle of maximum entropy, as shown in equation 3, offers an attractive solution to this task as it has several valuable attributes, both theoretical and practical. The principle states that, given a set of constraints, one should select the distribution with the highest entropy. Subsequently, this ensures that the chosen posterior distribution contains the least amount of information, thus only encoding the information present in the constraints.