Bayesian Learning
Explanation of Probabilistic Inference for Decision Support Systems
An automated explanation facility for Bayesian conditioning aimed at improving user acceptance of probability-based decision support systems has been developed. The domain-independent facility is based on an information processing perspective on reasoning about conditional evidence that accounts both for biased and normative inferences. Experimental results indicate that the facility is both acceptable to naive users and effective in improving understanding.
Evaluation of Uncertain Inference Models I: PROSPECTOR
Yadrick, Robert M., Perrin, Bruce M., Vaughan, David S., Holden, Peter D., Kempf, Karl G.
This paper examines the accuracy of the PROSPECTOR model for uncertain reasoning. PROSPECTOR's solutions for a large number of computer-generated inference networks were compared to those obtained from probability theory and minimum cross-entropy calculations. PROSPECTOR's answers were generally accurate for a restricted subset of problems that are consistent with its assumptions. However, even within this subset, we identified conditions under which PROSPECTOR's performance deteriorates.
A Randomized Approximation Algorithm of Logic Sampling
Chavez, R. Martin, Cooper, Gregory F.
In recent years, researchers in decision analysis and artificial intelligence (AI) have used Bayesian belief networks to build models of expert opinion. Using standard methods drawn from the theory of computational complexity, workers in the field have shown that the problem of exact probabilistic inference on belief networks almost certainly requires exponential computation in the worst ease [3]. We have previously described a randomized approximation scheme, called BN-RAS, for computation on belief networks [ 1, 2, 4]. We gave precise analytic bounds on the convergence of BN-RAS and showed how to trade running time for accuracy in the evaluation of posterior marginal probabilities. We now extend our previous results and demonstrate the generality of our framework by applying similar mathematical techniques to the analysis of convergence for logic sampling [7], an alternative simulation algorithm for probabilistic inference.
The Myth of Modularity in Rule-Based Systems
Heckerman, David, Horvitz, Eric J.
In this paper, we examine the concept of modularity, an often cited advantage of the ruled-based representation methodology. We argue that the notion of modularity consists of two distinct concepts which we call syntactic modularity and semantic modularity. We argue that when reasoning under certainty, it is reasonable to regard the rule-based approach as both syntactically and semantically modular. However, we argue that in the case of plausible reasoning, rules are syntactically modular but are rarely semantically modular. To illustrate this point, we examine a particular approach for managing uncertainty in rule-based systems called the MYCIN certainty factor model. We formally define the concept of semantic modularity with respect to the certainty factor model and discuss logical consequences of the definition. We show that the assumption of semantic modularity imposes strong restrictions on rules in a knowledge base. We argue that such restrictions are rarely valid in practical applications. Finally, we suggest how the concept of semantic modularity can be relaxed in a manner that makes it appropriate for plausible reasoning.
Interval Influence Diagrams
Fertig, Kenneth W., Breese, John S.
We describe a mechanism for performing probabilistic reasoning in influence diagrams using interval rather than point valued probabilities. We derive the procedures for node removal (corresponding to conditional expectation) and arc reversal (corresponding to Bayesian conditioning) in influence diagrams where lower bounds on probabilities are stored at each node. The resulting bounds for the transformed diagram are shown to be optimal within the class of constraints on probability distributions that can be expressed exclusively as lower bounds on the component probabilities of the diagram. Sequences of these operations can be performed to answer probabilistic queries with indeterminacies in the input and for performing sensitivity analysis on an influence diagram. The storage requirements and computational complexity of this approach are comparable to those for point-valued probabilistic inference mechanisms, making the approach attractive for performing sensitivity analysis and where probability information is not available. Limited empirical data on an implementation of the methodology are provided.
The Recovery of Causal Poly-Trees from Statistical Data
Poly-trees are singly connected causal networks in which variables may arise from multiple causes. This paper develops a method of recovering ply-trees from empirically measured probability distributions of pairs of variables. The method guarantees that, if the measured distributions are generated by a causal process structured as a ply-tree then the topological structure of such tree can be recovered precisely and, in addition, the causal directionality of the branches can be determined up to the maximum extent possible. The method also pinpoints the minimum (if any) external semantics required to determine the causal relationships among the variables considered.
The Rational and Computational Scope of Probabilistic Rule-Based Expert Systems
Belief updating schemes in artificial intelligence may be viewed as three dimensional languages, consisting of a syntax (e.g. probabilities or certainty factors), a calculus (e.g. Bayesian or CF combination rules), and a semantics (i.e. cognitive interpretations of competing formalisms). This paper studies the rational scope of those languages on the syntax and calculus grounds. In particular, the paper presents an endomorphism theorem which highlights the limitations imposed by the conditional independence assumptions implicit in the CF calculus. Implications of the theorem to the relationship between the CF and the Bayesian languages and the Dempster-Shafer theory of evidence are presented. The paper concludes with a discussion of some implications on rule-based knowledge engineering in uncertain domains.
Heuristic Search as Evidential Reasoning
BPS, the Bayesian Problem Solver, applies probabilistic inference and decision-theoretic control to flexible, resource-constrained problem-solving. This paper focuses on the Bayesian inference mechanism in BPS, and contrasts it with those of traditional heuristic search techniques. By performing sound inference, BPS can outperform traditional techniques with significantly less computational effort. Empirical tests on the Eight Puzzle show that after only a few hundred node expansions, BPS makes better decisions than does the best existing algorithm after several million node expansions
Approximations in Bayesian Belief Universe for Knowledge Based Systems
Jensen, Frank, Anderson, S. K.
When expert systems based on causal probabilistic networks (CPNs) reach a certain size and complexity, the "combinatorial explosion monster" tends to be present. We propose an approximation scheme that identifies rarely occurring cases and excludes these from being processed as ordinary cases in a CPN-based expert system. Depending on the topology and the probability distributions of the CPN, the numbers (representing probabilities of state combinations) in the underlying numerical representation can become very small. Annihilating these numbers and utilizing the resulting sparseness through data structuring techniques often results in several orders of magnitude of improvement in the consumption of computer resources. Bounds on the errors introduced into a CPN-based expert system through approximations are established. Finally, reports on empirical studies of applying the approximation scheme to a real-world CPN are given.
Probabilistic Interpretations for MYCIN's Certainty Factors
This paper examines the quantities used by MYCIN to reason with uncertainty, called certainty factors. It is shown that the original definition of certainty factors is inconsistent with the functions used in MYCIN to combine the quantities. This inconsistency is used to argue for a redefinition of certainty factors in terms of the intuitively appealing desiderata associated with the combining functions. It is shown that this redefinition accommodates an unlimited number of probabilistic interpretations. These interpretations are shown to be monotonic transformations of the likelihood ratio p(EIH)/p(El H). The construction of these interpretations provides insight into the assumptions implicit in the certainty factor model. In particular, it is shown that if uncertainty is to be propagated through an inference network in accordance with the desiderata, evidence must be conditionally independent given the hypothesis and its negation and the inference network must have a tree structure. It is emphasized that assumptions implicit in the model are rarely true in practical applications. Methods for relaxing the assumptions are suggested.