statement
Consensus statement on the credibility assessment of ML predictors
Aldieri, Alessandra, Gamage, Thiranja Prasad Babarenda, La Mattina, Antonino Amedeo, Li, Yi, Loewe, Axel, Pappalardo, Francesco, Italy, Marco Viceconti
The rapid integration of machine learning (ML) predictors into in silico medicine has revolutionized the estimation of quantities of interest (QIs) that are otherwise challenging to measure directly. However, the credibility of these predictors is critical, especially when they inform high-stakes healthcare decisions. This position paper presents a consensus statement developed by experts within the In Silico World Community of Practice. We outline twelve key statements forming the theoretical foundation for evaluating the credibility of ML predictors, emphasizing the necessity of causal knowledge, rigorous error quantification, and robustness to biases. By comparing ML predictors with biophysical models, we highlight unique challenges associated with implicit causal knowledge and propose strategies to ensure reliability and applicability. Our recommendations aim to guide researchers, developers, and regulators in the rigorous assessment and deployment of ML predictors in clinical and biomedical contexts.
Language Models Hallucinate, but May Excel at Fact Verification
Guan, Jian, Dodge, Jesse, Wadden, David, Huang, Minlie, Peng, Hao
Recent progress in natural language processing (NLP) owes much to remarkable advances in large language models (LLMs). Nevertheless, LLMs frequently "hallucinate," resulting in non-factual outputs. Our carefully designed human evaluation substantiates the serious hallucination issue, revealing that even GPT-3.5 produces factual outputs less than 25% of the time. This underscores the importance of fact verifiers in order to measure and incentivize progress. Our systematic investigation affirms that LLMs can be repurposed as effective fact verifiers with strong correlations with human judgments, at least in the Wikipedia domain. Surprisingly, FLAN-T5-11B, the least factual generator in our study, performs the best as a fact verifier, even outperforming more capable LLMs like GPT3.5 and ChatGPT. Delving deeper, we analyze the reliance of these LLMs on high-quality evidence, as well as their deficiencies in robustness and generalization ability. Our study presents insights for developing trustworthy generation models.
The Real Estate Agent-Modeling Users By Uncertain Reasoning
Two topics are treated here First, we present a user model pattcrncd after the stereotype approach (Rich, 1979) This model surpasses Rich's model with respect to its greater flexibility in the construction of user profiles, and its trcat,ment of positive and negative arguments. Second, we present an inference machine This machine treats uncertain knowledge in t,he form of evidence for and against the accuracy of a proposition. Assuming a homogeneous user group, systems developers were able to design a system to perform in accordance with the requirements and capabilities assumed for a partirulal type of user (implicit user modeling). With a heterogeneous user group, this is no longer possible. Since self-assessment,s usually render a distorted picture of the user and are not expected in a real consultative dialogue, they should not be specially required in man-machine communication.
Letters
And, "If Cohen is taken (overly) seriously, then every major MAD will prove generally useful. The authors relied heavily on the analysis of metaphor proffered by Earl MacCormic, and his distinction between epiphor and diaphor is useful in the analysis of metaphor in science. Readers of AI Magazine should be apprised, however, of other works on metaphor which may help clarify some of the issues. For one thing, there exists a considerable literature of experimental research on metaphor comprehension which bears on the claim that metaphors can be described as similarity statements. Suffice it to say that the scaling of the fruitfulness of metaphors by means of simple metrics of the features that are shared by the terms in a metaphor is not so straightforward as West and Travis imply Furthermore, the distinction between diaphor and epiphor is rather slippery, and the slipperiness is compounded by West and Travis' additional concept of "paraphor."
Mac made intelligent
I should like to lodge a complaint about your editorial standards in the article "An Assessment of Tools for Building Large KB Systems," by William Mettrey, in the winter 1987 [volume 9 number As a primary architect of CRL-Ops and a former KnowledgeCraft class instructor, I had to deal with the general public's misconceptions about forward versus backward chaining systems. Mr. Mettrey's article, in my opinion, is the type which generates the confusion that forward chaining rule systems cannot "backwards chain." This nonsensical view was held by the vast majority of our customers in the KC class. The section on Rule-Based inference implies that backward chaining is done only by Prolog in KC with its statement "by contrast, Knowledge-Craft implements backward chaining by supporting a version of Prolog." Any forward chaining rules system can efficiently implement constrained backward chaining by simply using a goal structure to search for the required knowledge.
Preference Handling -- An Introductory Tutorial
We present a tutorial introduction to the area of preference handling--one of the core issues in the design of any system that automates or supports decision making. The main goal of this tutorial is to provide a framework, or perspective, within which current work on preference handling--representation, reasoning, and elicitation--can be understood. Our intention is not to provide a technical description of the diverse methods used but rather to provide a general perspective on the problem and its varied solutions and to highlight central ideas and techniques. Hence an understanding of the various aspects of preference handling should be of great relevance to anyone attempting to build systems that act on behalf of users or simply support their decisions. This could be a shopping site that attempts to help us identify the most preferred item, an information search and retrieval engine that attempts to provide us with the most preferred pieces of information, or more sophisticated embedded agents such as robots, personal assistants, and so on.
The central thesis of my dissertation (Kocabas 1989)
I describe a formalism for organizing knowledge into such functional categories and some of its implementations. In this formalism, descriptive scientific knowledge is classified into seven categories. The categorization formalism allows complex propositions to be analyzed into their simple constituents; in turn, these constituents can be maintained in their categories. They can then be combined using a simple transformation function to form complex constructs such as frames and schemata. To demonstrate its viability, I implemented this knowledge organization in four computational models of scientific reasoning and discovery that operate in astronomy and particle physics, incorporating various methods of learning and theory revision.
Automated Theorem Proving: Theory and Practice A Review
ATP systems are used in a wide variety of domains: A mathematician might use the axioms of group theory to prove the conjecture that groups of order two are commutative; a management consultant might formulate axioms that describe how organizations grow and interact and, from these axioms, prove that organizational death rates decrease with age; or a frustrated teenager might formulate the jumbled faces of a Rubik's cube as a conjecture and prove, from axioms that describe legal changes to the cube's configuration, that the cube can be rearranged to the solution state. All these tasks can be performed by an ATP system, given an appropriate formulation of the problem as axioms, hypotheses, and a conjecture. Most commonly, ATP systems are embedded as components of larger, more complex software systems, and in this context, the ATP systems are required to autonomously solve subproblems that are generated by the overall system. To build a useful ATP system, several issues have to ...
AAAI News
Please Join Us for AAAI / IAAI 2013 in Bellevue, Washington! The Twenty-Seventh AAAI Conference on Artificial Intelligence (AAAI-13) will be held July 14-18, 2013, at the Hyatt Regency in Bellevue, Washington, USA. The Twenty-Fifth Conference on Innovative Applications of Artificial Intelligence (IAAI-13) will be collocated with AAAI-13, and will be held July 16-18 (see www.aaai.org/ The AAAI program cochairs for 2013, Marie desJardins (University of Maryland, Baltimore County, USA) and Michael Littman (Brown University, USA) will continue the innovations introduced during the past several years of the conference, including the four special technical tracks and the AI Subarea Spotlight track. An overview of the special tracks is included below.