An Empirical Comparison of Three Inference Methods

Several years ago, before learning much about methods for reasoning with uncertainty, I and my colleagues began work on a large expert system, called Pathfinder, that assists community pathologists with the diagnosis of lymph node pathology. Because the Dempster-Shafer theory of belief was quite popular in our research group at the time, we developed a inference method for our expert system inspired by this theory. The program performed fairly well in the opinion of the expert pathologist who provided the knowledge for the system. In the months following the initial development of Pathfinder, several of us in the research group began exploring other methods for reasoning under uncertainty. We identified the Bayesian approach as a candidate for a new inference procedure. We realized that the measures of uncertainty we assessed from the expert could be interpreted as probabilities and we implemented a new inference method-- a special case of Bayes' theorem. During this time, the expert was running cases through the program to test the system's diagnostic performance. One day, without telling him, we changed the inference procedure to the Bayesian approach. After running several cases with the new approach, the expert exclaimed, "What did you do to the program?