In this manuscript, we offer a gentle review of submodularity and supermodularity and their properties. We offer a plethora of submodular definitions; a full description of a number of example submodular functions and their generalizations; example discrete constraints; a discussion of basic algorithms for maximization, minimization, and other operations; a brief overview of continuous submodular extensions; and some historical applications. We then turn to how submodularity is useful in machine learning and artificial intelligence. This includes summarization, and we offer a complete account of the differences between and commonalities amongst sketching, coresets, extractive and abstractive summarization in NLP, data distillation and condensation, and data subset selection and feature selection. We discuss a variety of ways to produce a submodular function useful for machine learning, including heuristic hand-crafting, learning or approximately learning a submodular function or aspects thereof, and some advantages of the use of a submodular function as a coreset producer. We discuss submodular combinatorial information functions, and how submodularity is useful for clustering, data partitioning, parallel machine learning, active and semi-supervised learning, probabilistic modeling, and structured norms and loss functions.

This paper provides a unified perspective for the Kullback-Leibler (KL)-divergence and the integral probability metrics (IPMs) from the perspective of maximum likelihood density-ratio estimation (DRE). Both the KL-divergence and the IPMs are widely used in various fields in applications such as generative modeling. However, a unified understanding of these concepts has still been unexplored. In this paper, we show that the KL-divergence and the IPMs can be represented as maximal likelihoods differing only by sampling schemes, and use this result to derive a unified form of the IPMs and a relaxed estimation method. To develop the estimation problem, we construct an unconstrained maximum likelihood estimator to perform DRE with a stratified sampling scheme. We further propose a novel class of probability divergences, called the Density Ratio Metrics (DRMs), that interpolates the KL-divergence and the IPMs. In addition to these findings, we also introduce some applications of the DRMs, such as DRE and generative adversarial networks. In experiments, we validate the effectiveness of our proposed methods.

Probability for improved business decisions: Introduction, Combinatorics, Bayesian Inference, Distributions. Welcome to Probability and Statistics for Business and Data Science! In this course we cover what you need to know about probability and statistics to succeed in business and the data science field! This practical course will go over theory and implementation of statistics to real world problems. Each section has example problems, in course quizzes, and assessment tests.

While artificial-intelligence-based methods suffer from lack of transparency, rule-based methods dominate in safety-critical systems. Yet, the latter cannot compete with the first ones in robustness to multiple requirements, for instance, simultaneously addressing safety, comfort, and efficiency. Hence, to benefit from both methods they must be joined in a single system. This paper proposes a decision making and control framework, which profits from advantages of both the rule- and machine-learning-based techniques while compensating for their disadvantages. The proposed method embodies two controllers operating in parallel, called Safety and Learned. A rule-based switching logic selects one of the actions transmitted from both controllers. The Safety controller is prioritized every time, when the Learned one does not meet the safety constraint, and also directly participates in the safe Learned controller training. Decision making and control in autonomous driving is chosen as the system case study, where an autonomous vehicle learns a multi-task policy to safely cross an unprotected intersection. Multiple requirements (i.e., safety, efficiency, and comfort) are set for vehicle operation. A numerical simulation is performed for the proposed framework validation, where its ability to satisfy the requirements and robustness to changing environment is successfully demonstrated.

Bayesian bandit algorithms with approximate inference have been widely used in practice with superior performance. Yet, few studies regarding the fundamental understanding of their performances are available. In this paper, we propose a Bayesian bandit algorithm, which we call Generalized Bayesian Upper Confidence Bound (GBUCB), for bandit problems in the presence of approximate inference. Our theoretical analysis demonstrates that in Bernoulli multi-armed bandit, GBUCB can achieve $O(\sqrt{T}(\log T)^c)$ frequentist regret if the inference error measured by symmetrized Kullback-Leibler divergence is controllable. This analysis relies on a novel sensitivity analysis for quantile shifts with respect to inference errors. To our best knowledge, our work provides the first theoretical regret bound that is better than $o(T)$ in the setting of approximate inference. Our experimental evaluations on multiple approximate inference settings corroborate our theory, showing that our GBUCB is consistently superior to BUCB and Thompson sampling.

This paper addresses the problem of precisely estimating the parameters of a stochastic model corresponding to branching processes. A branching process is a stochastic process consisting of collections of random variables indexed by the natural numbers. Branching processes are often used to describe population models Jagers (1989) and Athreya and Ney (2012); for example, models in the population genetics showing the genetic drift Burden and Simon (2016) Chen et al. (2017). In contrast to statistical approaches, branching processes enable the study of the dynamics of cell evolution and, as a consistence, have become a popular approach to cancer cell evolution research West et al., 2016. However, particularly in the case of cancer cell evolution, as well as in branching processes in general, the ultimate extinction of a population often occurs Devroye (1998). It is for this reason that with the initial uniform distribution of parameters, branching processes models tend to yield unevenly distributed data consisting of sparse and dense regions. The stochastic nature of the data is an another obstacle in estimating the parameters of a branching processes model, especially in the case of cancer cell evolution Nagornov et al. (2021). Moreover, simulations, based on a model of cell mutations, population evolution, and tumor/cancer subpopulations, commonly lead to the emergence of many clones and rarely to the appearance of cancer cells.

Fuzzy Logic isn't often mentioned in the same room as Artificial Intelligence (AI). Pardon the pun, but most people find the idea of fuzzy logic to be fuzzy. However fuzzy logic has been working behind the scenes and bringing forth amazing technological advances for more than two decades. Fuzzy logic is a rule-based system that can rely on the practical experience of a data scientist or an expert. Fuzzy logic is a form of artificial intelligence, thus it is considered a subset of AI. Since it is performing a form of decision making, it can be included as a member of the AI family which includes Machine Learning and Deep Learning.

Generative Adversarial Networks (GANs) software is software for producing forgeries and imitations of data (aka synthetic data, fake data). Human beings have been making fakes, with good or evil intent, of almost everything they possibly can, since the beginning of the human race. Thus, perhaps not too surprisingly, GAN software has been widely used since it was first proposed in this amazingly recent 2014 paper. To gauge how widely GAN software has been used so far, see, for example, this 2019 article entitled "18 Impressive Applications of Generative Adversarial Networks (GANs)" Sounds (voices, music,…), Images (realistic pictures, paintings, drawings, handwriting, …), Text,etc. The forgeries can be tweaked so that they range from being very similar to the originals, to being whimsical exaggerations thereof.

Aqueous hydrolysis is used to chemically recycle polyethylene terephthalate (PET) due to the production of high-quality terephthalic acid (TPA), the PET monomer. PET hydrolysis depends on various reaction conditions including PET size, catalyst concentration, reaction temperature, etc. So, modeling PET hydrolysis by considering the effective factors can provide useful information for material scientists to specify how to design and run these reactions. It will save time, energy, and materials by optimizing the hydrolysis conditions. Machine learning algorithms enable to design models to predict output results. For the first time, 381 experimental data were gathered to model the aqueous hydrolysis of PET. Effective reaction conditions on PET hydrolysis were connected to TPA yield. The logistic regression was applied to rank the reaction conditions. Two algorithms were proposed, artificial neural network multilayer perceptron (ANN-MLP) and adaptive network-based fuzzy inference system (ANFIS). The dataset was divided into training and testing sets to train and test the models, respectively. The models predicted TPA yield sufficiently where the ANFIS model outperformed. R-squared (R2) and Root Mean Square Error (RMSE) loss functions were employed to measure the efficiency of the models and evaluate their performance.

Applications of neural networks have achieved great success in various fields. A major extension of the standard neural networks is to make them stochastic, namely, to make the output a random function of the input. In a broad sense, stochastic neural networks include neural networks trained with dropout (Srivastava et al., 2014; Gal & Ghahramani, 2016), Bayesian networks (Mackay, 1992), variational autoencoders (VAE) (Kingma & Welling, 2013), and generative adversarial networks (Goodfellow et al., 2014). There are many reasons why one wants to make a neural network stochastic. Two main reasons are (1) regularization and (2) distribution modeling.