This book is a collection of many of the seminal papers from the first decade of research in artificial intelligence in medicine (AIM). The editors state that the need for such a collection became evident when a two-day AIM tutorial was held at Stanford in 1980, following the annual national AIM research workshop. The 19 papers included in the book are each introduced by a short section written by the editors. Typically one page in length, these introductory sections are designed to place the paper into context in the field. In addition, the editors have included introductory and concluding chapters of their own.

Among the difficulties in evaluating AItype medical diagnosis systems are: the intermediate conclusions of the AI system need to be looked at in addition to the "final" answer; the "superhuman human" fallacy must be resisted; both pro-and anti-computer biases during evaluation must be guarded against; and methods for estimating how the approach will scale upwards to larger domains are needed We propose a type of Turing test for the evaluation problem, designed to provide some protection against the problems listed above We propose to measure both the accuracy of diagnosis and the structure of reasoning, the latter with a view to gauging how well the system will scale up A staple of many of the evaluations of AI systems that have so far been conducted (Colby, Hilf, Weber, 81 Kraemer, 1972; Yu et al, 1979) is a central idea from a well-known proposal to evaluate AI systems: The Turing Test (Turing, 1963) The meat of the idea is to see if a neutral observer, given a set of performances on a task, some by a machine and others by humans, but unlabelled as to authorship, could identify, better than chance, which were machine and which were human-produced. Note that this really attempts to answer the question, "DO we know how to design a machine to perform a task which until now required human intelligence?", The latter question subsumes the former in a sense: because the machine not performing well in comparison to a human would presumably increase the cost significantly. In this paper I follow tradition and consider the evaluation of AI systems for medical diagnosis from the viewpoint of the first question above. The proposed procedure is also a variant of Turing's Test.

Instead they have commonly assigned structure to a domain using statistical techniques such as Bayes' Theorem

The impact of a computer-based data management system on the completeness of clinical trial data was studied before and after the system's introduction in an oncology clinic. Physicians use the system, termed ONCOCIN, to record data during patient visits and to receive advice about treatment and tests required by experimental cancer protocols. Although ONCOCIN does not force the user to enter all data expected by the protocol, after its introduction there was improvement in the recording frequency of such data. The percentage of expected physical findings recorded increased from 74% to 91% (p .05),

ONCOCIN is a rule-based expert system to advise on cancer chemotherapy. Although shown to provide excellent advice, the program could not be easily adapted to critique a physician's treatment plan without incorporation of additional knowledge of the structure of experimental protocols. A separate effort to automate the encoding of new oncology protocols was impeded by the lack of structural organization in the knowledge base. In both cases, problems arose because ONCOCIN's knowledge representation scheme did not reflect the hierarchy of control knowledge inherent in oncology protocols. The limitations of current knowledge representation techniques in ONCOCIN are discussed. In ONCOCIN is a medical expert system that assists physicians In the treatment of cancer patients enrolled in chemotherapy protocols.

Optimal construction of expert systems demands a powerful interactive environment for knowledge base management by knowledge engineers. Key requirements include techniques for (a) examining existing information, (b) adding new knowledge and editing preexisting data structures, and (c) examining dynamic internal system behavior to facilitate debugging during consideration of actual cases.

Researchers in the development of medical expert systems have Increasingly recognized the Importance of explanation capabilities in encouraging the acceptance of their programs. One survey of potential users of medical advice systems has suggested that explanation may be the single most important capability of an acceptable clinical decision tool (16). Good explanations serve four functions in a consultation system: 111 they provide a method for examining the program's reasoning if errors arise when the system is being built; 121 they assure users that the reasoning is logical, thereby increasing user acceptance of the system; 131 they may persuade users that unexpected advice is appropriate; and 141 they can educate users in areas where their knowledge may be weak.

The ONCOCIN Project is supported by research grants from the National Library of Medicine, the Division of Research Resources of the NIH, the Office of Naval Re"arch, and the Henry J. Kaiser Family Foundation.

Reprinted by permission of the author. Published in the Proceedings of a Symposium on Computers in Medicine, Annual Meeting, California Medical Association, Anaheim, CA., February 1984. Alt;iough computing technology is playing an increasingly important role in medicine, systems designed to advise physicians on diagnosis or therapy selection have remained largely experimental to date. Despite diverse research efforts, and a literature on computer-aided diagnosis that has numbered over 1500 references in the last 20 years, clinical consultation programs have failed to achieve wide acceptance. The reasons for attempting to develop such systems are self-evident.

"Coming to Terms With the Computer" by Edward H. Shortliffe reprinted with permission from The Machine at the Bedside, Eds. You are asked to assist a major teaching hospital in the assessment of a large computer system that was installed 3 months ago to help with doctors' orders, laboratory test reporting, nursing schedules, and bed control. Because of mixed reviews of the system's effectiveness, the hospital has decided to bring in outside experts to assess the computer's strengths and weaknesses. The computer system was installed by a vendor of large-scale hospital information systems (HIS). The company had developed the programs over several years, but this is its first major commercial installation.