J. Barclay Adams, M.D., Ph.D. Associate Physician Department of Medicine Brigham and Women's Hospital Harvard Medical School Boston, Massachusetts 02115 Janice s. Aikins, Ph.D. Research Computer Scientist IBM Palo Alto Scientific Center 1530 Page Mill Road Palo Alto, California 94304 James s. Bennett, M.S. Senior Knowledge Engineer Teknowledge, Inc. 525 University Avenue Palo Alto, California 94301 Sharon Wraith Bennett, R.Ph.

Wraith, S. (see Bennett, S. SACON, 11,304, 312, 417, Swanson, D. B., 567

Steps toward a theoretic foundation for complex knowledge-based CAI.

Rephrases the question and gives examples of recognized responses.

Although the "MYCIN gang" has disbanded, the excitement of creating the constellation of programs described in this book is still evident. The legacy of this group is a set of results that others can build on. We are proud to have been active in the definition and development of MYCIN and are enriched by having worked with such an outstanding group of individuals.

In this book we have presented experimental evidence at many levels of detail for a diverse set of hypotheses. As indicated by the chapter and section headings, the major themes of the MYCIN work have many variations. In this final chapter we will try to summarize the most important results of the work presented. This recapitulation of the lessons learned should not be taken as a substitute for details in the sections themselves. We provide here an abstraction of the details, but hope it also constitutes a useful set of lessons on which others can build. The three main sections of this chapter will reiterate the main goals that provide the context for the experimental work; discuss the experimental results from each of the major parts of the book; and summarize the key questions we have been asked, or have asked ourselves, about the lessons we have learned. If we were to try to summarize in one word why MYCIN works as well as it does, that word would be flexibility.

This chapter describes an oncology protocol management system, named ONCOCIN after its domain of expertise (cancer therapy) and its historical debt to MYCIN. The program is actually a set of interrelated subsystems, the 1 primary ones being: 1. the Reasoner, a rule-based expert consultant that is the core of the system; and 2. the Interviewer, an interface program that controls a high-speed terminal and the interaction with the physicians using the system. The Interviewer is described in some detail in Chapter 32. This chapter describes the problem domain and the representation and control techniques used by the Reasoner. This chapter is based on an article originally appearing in Proceedings of the Seventh IJCAI, 1981, pp. Used by permission of" International Joint Conferences on Artificial Intelligence, Inc.; copies of the Proceedings are available from William Kaufmann, Inc., 95 First Street, l.os Ahos, CA 94022. Another program, the Intemctor, handles interprocess communication.

Effect of the Tutorial The tutorial experience had a small but significant effect on physicians' Demands and also produced a substantial increase in their knowledge about computing concepts. The results from the Demand-scale were of particular interest. Physicians apparently gained new insights from the tutorial into the potential use and capabilities of medical computing and increased their performance Demands accordingly. These opinions regarding the attributes of acceptable computing systems were surprisingly uniform across physician subgroups both before and after the tutorial. Our interpretation of this result is that physicians are serious about these Demands and that consultation systems are not likely to be clinically effective, regardless of the accuracy of their advice, until these capabilities have been incorporated.

Psychological work on memory, in particular by Bartlett (1932), has led the conclusion that people faced with a new situation use large amounts of highly structured knowledge acquired from previous experience. Bartlett used the word schema to refer to this phenomenon. Minsky (1975), his famous paper, proposed the notion of a frame as a fundamental structure used in natural language understanding, as well as in scene analysis. I will use the former term in the rest of this chapter, in spite of its general connotation. The main thesis defended by Bartlett was that the phenomena of memorization and remembering are both constructive and selective. The hypothesis has more recently been revived by psychologists working on discourse structure (Collins, 1978; Bransford and Franks, 1971; Kintsch, 1976). Various experiments performed on subjects who were told stories and then asked to describe what they remembered showed that people not only forget facts but add some. Moreover, they are unable to distinguish between what they have actually heard and what they have inferred. People hearing a story make assumptions, which they might revise or refine as more information comes in, either confirmatory or contradictory. Making such assumptions entails building (or retrieving) models of the expected text contents. A corollary of this process is that if the story adequately fits the model people have in mind, the story will be understood more easily. This chal)ter is based on a technical memo (HPP-79-25) from the Heuristic Programming lh( iect, l)cparmlent of Computer Science, Stanford University.

In August of 1980 Stanford hosted the annual Workshop on Artificial Intelligence in Medicine, and we organized a twoday tutorial program so that local physicians who were interested could learn about this emerging discipline. In addition, funding from the Henry J. Kaiser Family Foundation allowed us to support a questionnaire-based project to assess physicians' attitudes. Finally, a doctoral student in educational psychology, Randy Teach, joined the project that summer and brought with him much-needed skills in the areas of statistics, study design, and the use of computer-based statistical packages. The resulting study used the physicians who were attending the AIM tutorial as subjects, with a control group of M.D.'s drawn from the surrounding community. Chapter 34 summarizes the results and concludes with design recommendations derived from the data analysis. The reader is referred to that chapter for details; however, it is pertinent to reiterate here that a program's ability to give explanations for its reasoning was judged to be the single most important requirement for an advice-giving system in medicine. This observation accounts for our continued commitment to research on explanation, both in the ONCOCIN program (Langlotz and Shortliffe, 1983) and in current doctoral dissertations from the Heuristic Programming Project (Cooper, 1984; Kunz, 1984).