A few years ago, at the Princeton Club in Manhattan, I chanced on a memorable chat with the Harvard psychologist Steven Pinker. His spouse, the philosopher Rebecca Goldstein, with whom he was tagging along, had been invited onto a panel to discuss the conflict between religion and science and Einstein's so-called "God letter," which was being auctioned at Christie's. Pinker had recently published Enlightenment Now: The Case for Reason, Science, Humanism, and Progress. I was eager to pepper him with questions, mainly on religion, rationality, and evolutionary psychology. I remember I wanted Pinker's take on something Harvey Whitehouse, one of the founders of the cognitive science of religion, told me in an interview--that my own little enlightenment, of becoming an atheist in college, was probably mostly a product of merely changing my social milieu. I wasn't so much moved by rational arguments against the ethics and existence of God but by being distanced from my old life and meeting new, non-religious friends. I recall Pinker almost pouncing on that argument, defending reason's power to change our minds. He noted that people especially high in "intellectance," a personality trait now more commonly called "openness to experience," tend to be more curious, intelligent, and willing to entertain new ideas. I still think that Pinker's way of seeing things made more sense of my experience in those heady days. I really was, for the first time, trying my best to think things through, and it was exhilarating. We talked until the event staff shelved the wine, and parted ways at a chilly midtown intersection.

Model misspecification can create significant challenges for the implementation of probabilistic models, and this has led to development of a range of inference methods which directly account for this issue. However, whether these more involved methods are required will depend on whether the model is really misspecified, and there is a lack of generally applicable methods to answer this question. One set of tools which can help are goodness-of-fit tests, where we test whether a dataset could have been generated by a fixed distribution. Kernel-based tests have been developed to for this problem, and these are popular due to their flexibility, strong theoretical guarantees and ease of implementation in a wide range of scenarios. In this paper, we extend this line of work to the more challenging composite goodness-of-fit problem, where we are instead interested in whether the data comes from any distribution in some parametric family. This is equivalent to testing whether a parametric model is well-specified for the data.

The construction sector constitutes one of the most important sectors in the economy of any country. Many construction projects experience time and cost overruns due to the wrong choice of contractors. In this paper, the feedback integrated multi-criteria group decision support model for contractor selection was proposed. The proposed model consists of two modules; technical evaluation module and financial evaluation module. The technical evaluation module is employed to screen out the contractors to a smaller set of acceptable contractors and the functionality of the module is based on the Fuzzy Analytic Hierarchy Process (FAHP).

False alerts due to misconfigured/ compromised IDS in ICS networks can lead to severe economic and operational damage. To solve this problem, research has focused on leveraging deep learning techniques that help reduce false alerts. However, a shortcoming is that these works often require or implicitly assume the physical and cyber sensors to be trustworthy. Implicit trust of data is a major problem with using artificial intelligence or machine learning for CPS security, because during critical attack detection time they are more at risk, with greater likelihood and impact, of also being compromised. To address this shortcoming, the problem is reframed on how to make good decisions given uncertainty. Then, the decision is detection, and the uncertainty includes whether the data used for ML-based IDS is compromised. Thus, this work presents an approach for reducing false alerts in CPS power systems by dealing uncertainty without the knowledge of prior distribution of alerts. Specifically, an evidence theoretic based approach leveraging Dempster Shafer combination rules are proposed for reducing false alerts. A multi-hypothesis mass function model is designed that leverages probability scores obtained from various supervised-learning classifiers. Using this model, a location-cum-domain based fusion framework is proposed and evaluated with different combination rules, that fuse multiple evidence from inter-domain and intra-domain sensors. The approach is demonstrated in a cyber-physical power system testbed with Man-In-The-Middle attack emulation in a large-scale synthetic electric grid. For evaluating the performance, plausibility, belief, pignistic, etc. metrics as decision functions are considered. To improve the performance, a multi-objective based genetic algorithm is proposed for feature selection considering the decision metrics as the fitness function.

Generative Flow Networks (GFlowNets) have been introduced as a method to sample a diverse set of candidates in an active learning context, with a training objective that makes them approximately sample in proportion to a given reward function. In this paper, we show a number of additional theoretical properties of GFlowNets. They can be used to estimate joint probability distributions and the corresponding marginal distributions where some variables are unspecified and, of particular interest, can represent distributions over composite objects like sets and graphs. GFlowNets amortize the work typically done by computationally expensive MCMC methods in a single but trained generative pass. They could also be used to estimate partition functions and free energies, conditional probabilities of supersets (supergraphs) given a subset (subgraph), as well as marginal distributions over all supersets (supergraphs) of a given set (graph). We introduce variations enabling the estimation of entropy and mutual information, sampling from a Pareto frontier, connections to reward-maximizing policies, and extensions to stochastic environments, continuous actions and modular energy functions.

We demonstrate that the space of intuitionistic fuzzy values (IFVs) with the linear order based on a score function and an accuracy function has the same algebraic structure as the one induced by the linear order based on a similarity function and an accuracy function. By introducing a new operator for IFVs via the linear order based on a score function and an accuracy function, we present that such an operator is a strong negation on IFVs. Moreover, we propose that the space of IFVs is a complete lattice and a Kleene algebra with the new operator. We also observe that the topological space of IFVs with the order topology induced by the above two linear orders is not separable and metrizable but compact and connected. From exactly new perspectives, our results partially answer three open problems posed by Atanassov [Intuitionistic Fuzzy Sets: Theory and Applications, Springer, 1999] and [On Intuitionistic Fuzzy Sets Theory, Springer, 2012]. Furthermore, we construct an isomorphism between the spaces of IFVs and q-rung orthopedic fuzzy values (q-ROFVs) under the corresponding linear orders. Meanwhile, we introduce the concept of the admissible similarity measures with particular orders for IFSs, extending the previous definition of the similarity measure for IFSs, and construct an admissible similarity measure with the linear order based on a score function and an accuracy function, which is effectively applied to a pattern recognition problem about the classification of building materials.

If you've ever searched for evaluation metrics to assess model accuracy, chances are that you found many different options to choose from. Accuracy is in some sense the holy grail of prediction so it's not at all surprising that the machine learning community spends a lot time thinking about it. In a world where more and more high-stake decisions are being automated, model accuracy is in fact a very valid concern. But does this recipe for model evaluation seem like a sound and complete approach to automated decision-making? Some would argue that we need to pay more attention to model uncertainty. No matter how many times you have cross-validated your model, the loss metric that it is being optimized against as well as its parameters and predictions remain inherently random variables.

Forecasting driving behavior or other sensor measurements is an essential component of autonomous driving systems. Often real-world multivariate time series data is hard to model because the underlying dynamics are nonlinear and the observations are noisy. In addition, driving data can often be multimodal in distribution, meaning that there are distinct predictions that are likely, but averaging can hurt model performance. To address this, we propose the Switching Recurrent Kalman Network (SRKN) for efficient inference and prediction on nonlinear and multi-modal time-series data. The model switches among several Kalman filters that model different aspects of the dynamics in a factorized latent state. We empirically test the resulting scalable and interpretable deep state-space model on toy data sets and real driving data from taxis in Porto. In all cases, the model can capture the multimodal nature of the dynamics in the data.

Deep Neural Networks (DNNs) have performed admirably in classification tasks. However, the characterization of their classification uncertainties, required for certain applications, has been lacking. In this work, we investigate the issue by assessing DNNs' ability to estimate conditional probabilities and propose a framework for systematic uncertainty characterization. Denoting the input sample as x and the category as y, the classification task of assigning a category y to a given input x can be reduced to the task of estimating the conditional probabilities p(y|x), as approximated by the DNN at its last layer using the softmax function. Since softmax yields a vector whose elements all fall in the interval (0, 1) and sum to 1, it suggests a probabilistic interpretation to the DNN's outcome. Using synthetic and real-world datasets, we look into the impact of various factors, e.g., probability density f(x) and inter-categorical sparsity, on the precision of DNNs' estimations of p(y|x), and find that the likelihood probability density and the inter-categorical sparsity have greater impacts than the prior probability to DNNs' classification uncertainty.

We start by introducing a simple example to illustrate the kind of problems we are interested in solving. Consider the problem of estimating a parameter θ using data from a small experiment and a prior distribution constructed from similar previous experiments. The specific problem description is borrowed from [3]: Estimating the risk of tumor in a group of rats. In the evaluation of drugs for possible clinical application, studies are routinely performed on rodents. For a particular study drawn from the statistical literature, suppose the immediate aim is to estimate θ, the probability of tumor in a population of female laboratory rats of type'F344' that receive a zero dose of the drug (a control group). The data show that 4 out of 14 rats developed endometrial stromal polyps (a kind of tumor). Typically, the mean and standard deviation of underlying tumor risks are not available. Rather, historical data are available on previous experiments on similar groups of rats. In the rat tumor example, the historical data were in fact a set of observations of tumor incidence in 70 groups of rats (table 1).