Target detection in hyperspectral imagery is the process of locating pixels from an image which are likely to contain target, typically done by comparing one or more spectra for the desired target material to each pixel in the image. Target identification is the process of target detection incorporating an additional process to identify more specifically the material that is present in each pixel that scored high in detection. Detection is generally a 2-class problem of target vs. background, and identification is a many class problem including target, background, and additional know materials. The identification process we present is probabilistic and hierarchical which provides transparency to the process and produces trustworthy output. In this paper we show that target identification has a much lower false alarm rate than detection alone, and provide a detailed explanation of a robust identification method using probabilistic hierarchical classification that handles the vague categories of materials that depend on users which are different than the specific physical categories of chemical constituents. Identification is often done by comparing mixtures of materials including the target spectra to mixtures of materials that do not include the target spectra, possibly with other steps. (band combinations, feature checking, background removal, etc.) Standard linear regression does not handle these problems well because the number of regressors (identification spectra) is greater than the number of feature variables (bands), and there are multiple correlated spectra. Our proposed method handles these challenges efficiently and provides additional important practical information in the form of hierarchical probabilities computed from Bayesian model averaging.

In this paper we provide classification In this paper we will be working hyperspectral pixel data methods for determining crop type in the USGS collected using the NASA Hyperion satellite [3] and organized GHISACONUS data, which contains around 7,000 pixel spectra and meticulously labeled by the USGS. This data, available from the five major U.S. agricultural crops (winter wheat, online from the USGS as the Global Hyperspectral Imaging rice, corn, soybeans, and cotton) collected by the NASA Spectral-library of Agricultural crops for Conterminous United Hyperion satellite, and includes the spectrum, geolocation, States (GHISACONUS) [4], is a library of 6,988 spectra, each crop type, and stage of growth for each pixel. We apply of which is labeled as one of the five major agricultural crops standard LDA and QDA as well as Bayesian custom versions (e.g., winter wheat, rice, corn, soybeans, and cotton) collected that compute the joint probability of crop type and stage, and between 2008 and 2015. The locations for the spectra in the then the marginal probability for crop type, outperforming GHISACONUS library are shown in Figure 1.

Causal Machine Learning (CausalML) is an umbrella term for machine learning methods that formalize the data-generation process as a structural causal model (SCM). This perspective enables us to reason about the effects of changes to this process (interventions) and what would have happened in hindsight (counterfactuals). We categorize work in CausalML into five groups according to the problems they address: (1) causal supervised learning, (2) causal generative modeling, (3) causal explanations, (4) causal fairness, and (5) causal reinforcement learning. We systematically compare the methods in each category and point out open problems. Further, we review data-modality-specific applications in computer vision, natural language processing, and graph representation learning. Finally, we provide an overview of causal benchmarks and a critical discussion of the state of this nascent field, including recommendations for future work.

Normalizing flows, diffusion normalizing flows and variational autoencoders are powerful generative models. This chapter provides a unified framework to handle these approaches via Markov chains. We consider stochastic normalizing flows as a pair of Markov chains fulfilling some properties and show how many state-of-the-art models for data generation fit into this framework. Indeed numerical simulations show that including stochastic layers improves the expressivity of the network and allows for generating multimodal distributions from unimodal ones. The Markov chains point of view enables us to couple both deterministic layers as invertible neural networks and stochastic layers as Metropolis-Hasting layers, Langevin layers, variational autoencoders and diffusion normalizing flows in a mathematically sound way. Our framework establishes a useful mathematical tool to combine the various approaches.

Examples can be found in all areas of engineering and science, for instance plant breeding (Hunter & McClosky, 2016), aircraft maintenance (Mattila & Virtanen, 2014) or electric network planning (Dutrieux et al., 2015b). In this work, we choose to focus on Bayesian optimization algorithms, which use probabilistic models to make predictions about the functions to be optimized. One of the classical algorithms for Bayesian deterministic multi-objective optimization is the ParEGO algorithm (Knowles, 2006). It relies on a scalarization approach to extend the very popular single-objective Efficient Global Optimization (EGO) algorithm (Jones et al., 1998), based on the Expected Improvement (EI) criterion. Another scalarization approach, called multi-attribute Knowledge Gradient (Astudillo & Frazier, 2017), uses the Knowledge Gradient (KG) criterion instead of the EI criterion.

In this article we consider Bayesian inference associated to deep neural networks (DNNs) and in particular, trace-class neural network (TNN) priors which were proposed by Sell et al. [39]. Such priors were developed as more robust alternatives to classical architectures in the context of inference problems. For this work we develop multilevel Monte Carlo (MLMC) methods for such models. MLMC is a popular variance reduction technique, with particular applications in Bayesian statistics and uncertainty quantification. We show how a particular advanced MLMC method that was introduced in [4] can be applied to Bayesian inference from DNNs and establish mathematically, that the computational cost to achieve a particular mean square error, associated to posterior expectation computation, can be reduced by several orders, versus more conventional techniques. To verify such results we provide numerous numerical experiments on model problems arising in machine learning. These include Bayesian regression, as well as Bayesian classification and reinforcement learning.

This work in progress aims to provide a unified introduction to statistical learning, building up slowly from classical models like the GMM and HMM to modern neural networks like the VAE and diffusion models. There are today many internet resources that explain this or that new machine-learning algorithm in isolation, but they do not (and cannot, in so brief a space) connect these algorithms with each other or with the classical literature on statistical models, out of which the modern algorithms emerged. Also conspicuously lacking is a single notational system which, although unfazing to those already familiar with the material (like the authors of these posts), raises a significant barrier to the novice's entry. Likewise, I have aimed to assimilate the various models, wherever possible, to a single framework for inference and learning, showing how (and why) to change one model into another with minimal alteration (some of them novel, others from the literature). Some background is of course necessary. I have assumed the reader is familiar with basic multivariable calculus, probability and statistics, and linear algebra. The goal of this book is certainly not completeness, but rather to draw a more or less straight-line path from the basics to the extremely powerful new models of the last decade. The goal then is to complement, not replace, such comprehensive texts as Bishop's \emph{Pattern Recognition and Machine Learning}, which is now 15 years old.

Bayesian optimization is a popular framework for the optimization of black box functions. Multifidelity methods allows to accelerate Bayesian optimization by exploiting low-fidelity representations of expensive objective functions. Popular multifidelity Bayesian strategies rely on sampling policies that account for the immediate reward obtained evaluating the objective function at a specific input, precluding greater informative gains that might be obtained looking ahead more steps. This paper proposes a non-myopic multifidelity Bayesian framework to grasp the long-term reward from future steps of the optimization. Our computational strategy comes with a two-step lookahead multifidelity acquisition function that maximizes the cumulative reward obtained measuring the improvement in the solution over two steps ahead. We demonstrate that the proposed algorithm outperforms a standard multifidelity Bayesian framework on popular benchmark optimization problems.

With the increasing abundance of pretrained models in recent years, the problem of selecting the best pretrained checkpoint for a particular downstream classification task has been gaining increased attention. Although several methods have recently been proposed to tackle the selection problem (e.g. LEEP, H-score), these methods resort to applying heuristics that are not well motivated by learning theory. In this paper we present PACTran, a theoretically grounded family of metrics for pretrained model selection and transferability measurement. We first show how to derive PACTran metrics from the optimal PAC-Bayesian bound under the transfer learning setting. We then empirically evaluate three metric instantiations of PACTran on a number of vision tasks (VTAB) as well as a language-and-vision (OKVQA) task. An analysis of the results shows PACTran is a more consistent and effective transferability measure compared to existing selection methods.

Semantic simultaneous localization and mapping is a subject of increasing interest in robotics and AI that directly influences the autonomous vehicles industry, the army industries, and more. One of the challenges in this field is to obtain object classification jointly with robot trajectory estimation. Considering view-dependent semantic measurements, there is a coupling between different classes, resulting in a combinatorial number of hypotheses. A common solution is to prune hypotheses that have a sufficiently low probability and to retain only a limited number of hypotheses. However, after pruning and renormalization, the updated probability is overconfident with respect to the original probability. This is especially problematic for systems that require high accuracy. If the prior probability of the classes is independent, the original normalization factor can be computed efficiently without pruning hypotheses. To the best of our knowledge, this is the first work to present these results. If the prior probability of the classes is dependent, we propose a lower bound on the normalization factor that ensures cautious results. The bound is calculated incrementally and with similar efficiency as in the independent case. After pruning and updating based on the bound, this belief is shown empirically to be close to the original belief.