Bayesian Learning
Uncertainty in Artificial Intelligence
The Fourth Uncertainty in Artificial Intelligence workshop was held 19-21 August 1988. The workshop featured significant developments in application of theories of representation and reasoning under uncertainty. A recurring idea at the workshop was the need to examine uncertainty calculi in the context of choosing representation, inference, and control methodologies. The effectiveness of these choices in AI systems tends to be best considered in terms of specific problem areas. These areas include automated planning, temporal reasoning, computer vision, medical diagnosis, fault detection, text analysis, distributed systems, and behavior of nonlinear systems. Influence diagrams are emerging as a unifying representation, enabling tool development. Interest and results in uncertainty in AI are growing beyond the capacity of a workshop format.
Decision theory in expert systems and artificial intelligence
Horvitz, E. J. | Breese, J. S. | Henrion, M.
Despite their different perspectives, artificial intelligence (AI) and the disciplines of decision science have common roots and strive for similar goals. This paper surveys the potential for addressing problems in representation, inference, knowledge engineering, and explanation within the decision-theoretic framework. Recent analyses of the restrictions of several traditional AI reasoning techniques, coupled with the development of more tractable and expressive decision-theoretic representation and inference strategies, have stimulated renewed interest in decision theory and decision analysis. We describe early experience with simple probabilistic schemes for automated reasoning, review the dominant expert-system paradigm, and survey some recent research at the crossroads of AI and decision science. In particular, we present the belief network and influence diagram representations.
Bayesian classification
Cheeseman, P. | Self, M. | Kelly, J. | Stutz, J.
This paper describes a Bayesian technique for unsupervised classification of data and its computer implementation, AutoClass. Given real valued or discrete data, AutoClass determines the most probable number of classes present in the data, the most probable descriptions of those classes, and each object's probability of membership in each class. The program performs as well as or better than other automatic classification systems when run on the same data and contains no ad hoc similarity measures or stopping criteria. AutoClass has been applied to several databases in which it has discovered classes representing previously unsuspected phenomena.
Thinking Backward for Knowledge Acquisition
Schachter, Ross D., Heckerman, David
This article examines the direction in which knowledge bases are constructed for diagnosis and decision making. When building an expert system, it is traditional to elicit knowledge from an expert in the direction in which the knowledge is to be applied, namely, from observable evidence toward unobservable hypotheses. However, experts usually find it simpler to reason in the opposite direction-from hypotheses to unobservable evidence-because this direction reflects causal relationships. Therefore, we argue that a knowledge base be constructed following the expert's natural reasoning direction, and then reverse the direction for use. This choice of representation direction facilitates knowledge acquisition in deterministic domains and is essential when a problem involves uncertainty. We illustrate this concept with influence diagrams, a methodology for graphically representing a joint probability distribution. Influence diagrams provide a practical means by which an expert can characterize the qualitative and quantitative relationships among evidence and hypotheses in the apporiate direction. Once constructed, the relationships can easily be reserved into the less intuitive direction in order to perform inference inference and diagnosis. In this way, knowledge acquisition is made cognitively simple; the machine carries the burden of translating the representation.
Decision analysis: a Bayesian approach
Chapman and Hall. See also: Influence diagrams for Bayesian decision analysis, European Journal of Operational Research, Volume 40, Issue 3, 15 June 1989, Pages 363–376 (http://www.sciencedirect.com/science/article/pii/0377221789904293). Bayesian Decision Analysis: Principles and Practice, Cambridge University Press, 2010 (https://books.google.com/books/about/Bayesian_Decision_Analysis.html?id=O1lXnQAACAAJ).
Fusion, propagation, and structuring in belief networks
Belief networks are directed acyclic graphs in which the nodes represent propositions (or variables), the arcs signify direct dependencies between the linked propositions, and the strengths of these dependencies are quantified by conditional probabilities. A network of this sort can be used to represent the generic knowledge of a domain expert, and it turns into a computational architecture if the links are used not merely for storing factual knowledge but also for directing and activating the data flow in the computations which manipulate this knowledge. The first part of the paper deals with the task of fusing and propagating the impacts of new information through the networks in such a way that, when equilibrium is reached, each proposition will be assigned a measure of belief consistent with the axioms of probability theory. It is shown that if the network is singly connected (e.g. The second part of the paper deals with the problem of finding a tree-structured representation for a collection of probabilistically coupled propositions using auxiliary (dummy) variables, colloquially called “hidden causes.”
In defense of probability
In Defense of Probability Peter Cheeseman SRI International 333 Ravenswood Ave., Menlo Park, California 94025 Abstract In this paper, it is argued that probability theory, when used correctly, is suffrcient for the task of reasoning under uncertainty. Since numerous authors have rejected probability as inadequate for various reasons, the bulk of the paper is aimed at refuting these claims and indicating the scources of error. In particular, the definition of probability as a measure of belief rather than a frequency ratio is advocated, since a frequency interpretation of probability drastically restricts the domain of applicability. Other sources of error include the confusion between relative and absolute probability, the distinction between probability and the uncertainty of that probability. Also, the interaction of logic and probability is discusses and it is argued that many extensions of logic, such as "default logic" are better understood in a probabilistic framework. The main claim of this paper is that the numerous schemes for representing and reasoning about uncertainty that have appeared in the AI literature are unnecessary--probability is all that is needed. 1 Introduction A glance through any major AI publication shows that an overwhelming proportion of papers are concerned with what might be described as the logical approach to inference and knowledge representation. It now widely accepted that many knowledge representations can be mapped into (first order) predicate calculus, and the corresponding inference procedures can be reduced to a type of controlled logical deduction. However, examples of human reasoning (judgements) are full of such terms as "probably", "most", "usually" etc., showing that many patterns of human reasoning are not logical in form, but intrinsically probabilistic. The claim that many patterns of human reasoning are probabilistic does not mean that the underlying "logic" of such patterns cannot be axiomatized. On the contrary, a basis for such an axiomatization is given in section 3. The claim is that when such an exercise is performed, the resulting patterns of inference are different in form from those found in analogous logical deductions.