This chapter briefly outlines the difference between traditional instructional programs and ICAI. It then illustrates how GUIDON makes contributions in areas important to medical CAl: interacting with the student in a mixed-initiative dialogue (including the problems of feedback and realism), teaching problem-solving strategies, and assembling a computerbased curriculum. In evaluating GUIDON's performance, one can see the value in the basic idea of formalizing teaching knowledge in procedures that are separate from the knowledge to be taught. However, the program is inherently limited by the MYCIN knowledge base. The rule set is poorly structured, does not contain pathophysiological knowledge for justifying the diagnostic associations, and does not explicitly state the strategies for gathering information and focusing on hypotheses.

Since the early 1970s, researchers in computer-based medical reasoning have begun to recognize the potential benefits of applying symbolic reasoning techniques in clinical domains (see Chapter 3). One such research group is the Heuristic Programming Project at Stanford University. The first medical reasoning program developed by the project, known as the MYCIN system (Shortliffe, 1976), adopted symbolic processing techniques largely in response to a conviction that computer-based consultation systems, in order to be accepted by physicians, should be able to explain how and why a particular conclusion has been derived. Such systems should also be able to incorporate, organize, manipulate, and update large quantities of medical knowledge. Subsequently, a series of additional medical application programs using MYCIN's techniques has been created.

Turing's test (see evaluation) smart instruments, 345

Chapter 10 Chapter 11 Chapter 12 Using Rules The Evolution of MYCIN's Rule Form The Structure of the MYCIN System William van Melle Details of the Consultation System Edward H. Shortliffe Details of the Revised Therapy Algorithm WiUiam J. Clancey Building a Knowledge Base Knowledge Engineering Completeness and Consistency in a Rule-Based System Motoi Suwa, A. Carlisle Scott, and Edward H. Shortliffe Interactive Transfer of Expertise Randall Davis Reasoning Under Uncertainty Uncertainty and Evidential Support A Model of Inexact Reasoning in Medicine Edward H. Shortliffe and Bruce G. Buchanan Probabilistic Reasoning and Certainty Factors J. Barclay Adams 55 67 78 133 149 159 171 209 233 263

Steps toward a theoretic foundation for complex knowledge-based CAI.

Rephrases the question and gives examples of recognized responses.

In this book we have presented experimental evidence at many levels of detail for a diverse set of hypotheses. As indicated by the chapter and section headings, the major themes of the MYCIN work have many variations. In this final chapter we will try to summarize the most important results of the work presented. This recapitulation of the lessons learned should not be taken as a substitute for details in the sections themselves. We provide here an abstraction of the details, but hope it also constitutes a useful set of lessons on which others can build. The three main sections of this chapter will reiterate the main goals that provide the context for the experimental work; discuss the experimental results from each of the major parts of the book; and summarize the key questions we have been asked, or have asked ourselves, about the lessons we have learned. If we were to try to summarize in one word why MYCIN works as well as it does, that word would be flexibility.

This chapter describes an oncology protocol management system, named ONCOCIN after its domain of expertise (cancer therapy) and its historical debt to MYCIN. The program is actually a set of interrelated subsystems, the 1 primary ones being: 1. the Reasoner, a rule-based expert consultant that is the core of the system; and 2. the Interviewer, an interface program that controls a high-speed terminal and the interaction with the physicians using the system. The Interviewer is described in some detail in Chapter 32. This chapter describes the problem domain and the representation and control techniques used by the Reasoner. This chapter is based on an article originally appearing in Proceedings of the Seventh IJCAI, 1981, pp. Used by permission of" International Joint Conferences on Artificial Intelligence, Inc.; copies of the Proceedings are available from William Kaufmann, Inc., 95 First Street, l.os Ahos, CA 94022. Another program, the Intemctor, handles interprocess communication.

Psychological work on memory, in particular by Bartlett (1932), has led the conclusion that people faced with a new situation use large amounts of highly structured knowledge acquired from previous experience. Bartlett used the word schema to refer to this phenomenon. Minsky (1975), his famous paper, proposed the notion of a frame as a fundamental structure used in natural language understanding, as well as in scene analysis. I will use the former term in the rest of this chapter, in spite of its general connotation. The main thesis defended by Bartlett was that the phenomena of memorization and remembering are both constructive and selective. The hypothesis has more recently been revived by psychologists working on discourse structure (Collins, 1978; Bransford and Franks, 1971; Kintsch, 1976). Various experiments performed on subjects who were told stories and then asked to describe what they remembered showed that people not only forget facts but add some. Moreover, they are unable to distinguish between what they have actually heard and what they have inferred. People hearing a story make assumptions, which they might revise or refine as more information comes in, either confirmatory or contradictory. Making such assumptions entails building (or retrieving) models of the expected text contents. A corollary of this process is that if the story adequately fits the model people have in mind, the story will be understood more easily. This chal)ter is based on a technical memo (HPP-79-25) from the Heuristic Programming lh( iect, l)cparmlent of Computer Science, Stanford University.

In August of 1980 Stanford hosted the annual Workshop on Artificial Intelligence in Medicine, and we organized a twoday tutorial program so that local physicians who were interested could learn about this emerging discipline. In addition, funding from the Henry J. Kaiser Family Foundation allowed us to support a questionnaire-based project to assess physicians' attitudes. Finally, a doctoral student in educational psychology, Randy Teach, joined the project that summer and brought with him much-needed skills in the areas of statistics, study design, and the use of computer-based statistical packages. The resulting study used the physicians who were attending the AIM tutorial as subjects, with a control group of M.D.'s drawn from the surrounding community. Chapter 34 summarizes the results and concludes with design recommendations derived from the data analysis. The reader is referred to that chapter for details; however, it is pertinent to reiterate here that a program's ability to give explanations for its reasoning was judged to be the single most important requirement for an advice-giving system in medicine. This observation accounts for our continued commitment to research on explanation, both in the ONCOCIN program (Langlotz and Shortliffe, 1983) and in current doctoral dissertations from the Heuristic Programming Project (Cooper, 1984; Kunz, 1984).