Situation
Report 81 09 Evaluating Expert Systems . Stanford Edward H. Jul 1981
This paper is the author's contribution to Chapter 6. in the volume BU:LDIN3 EXPERT SYSTEMS, edited by R. Hayes-Roth, D. Lenat, and D. Waterman:4 The full article is entitled "Evaluation of expert systems: issues and case studies", and is authored by J. Gaschnig, P. Klahr, H. Pople, E. Shortliffe. The volume is the result of a Workshop on Expert Systems held in San Diego in August 1980 and sponsored by the Rand Corporation, ARPA, and the NSF. Parts of Chapters 7 & 8. Reprinted with permission. In this section we define ma,ly of the parameters that determine an appropriate design for an evaluation experiment. When one examines the literature on computer performance evaluation, it is clear that the term is used with a variety of meanings depending upon the;.-c.3pective of tne authors. Each tends to focus on the specific performance issues that have been most central to the design of the system in question. Utner aspects warranting formal evaluation are often ignored.
RoboCup Rescue Robot and Simulation Leagues
Akin, H. Levent (Bogazici University) | Ito, Nobuhiro (Aichi Institute of Technology) | Jacoff, Adam (National Institute of Standards and Technology) | Kleiner, Alexander (Linköping University) | Pellenz, Johannes (V&R Vision &) | Visser, Arnoud (Robotics GmbH)
The RoboCup Rescue Robot and Simulation competitions have been held since 2000. The experience gained during these competitions has increased the maturity level of the field, which allowed deploying robots after real disasters (for example, Fukushima Daiichi nuclear disaster). This article provides an overview of these competitions and highlights the state of the art and the lessons learned.
Detecting Events and Patterns in Large-Scale User Generated Textual Streams with Statistical Learning Methods
A vast amount of textual web streams is influenced by events or phenomena emerging in the real world. The social web forms an excellent modern paradigm, where unstructured user generated content is published on a regular basis and in most occasions is freely distributed. The present Ph.D. Thesis deals with the problem of inferring information - or patterns in general - about events emerging in real life based on the contents of this textual stream. We show that it is possible to extract valuable information about social phenomena, such as an epidemic or even rainfall rates, by automatic analysis of the content published in Social Media, and in particular Twitter, using Statistical Machine Learning methods. An important intermediate task regards the formation and identification of features which characterise a target event; we select and use those textual features in several linear, non-linear and hybrid inference approaches achieving a significantly good performance in terms of the applied loss function. By examining further this rich data set, we also propose methods for extracting various types of mood signals revealing how affective norms - at least within the social web's population - evolve during the day and how significant events emerging in the real world are influencing them. Lastly, we present some preliminary findings showing several spatiotemporal characteristics of this textual information as well as the potential of using it to tackle tasks such as the prediction of voting intentions.
Notes on a New Philosophy of Empirical Science
This book presents a methodology and philosophy of empirical science based on large scale lossless data compression. In this view a theory is scientific if it can be used to build a data compression program, and it is valuable if it can compress a standard benchmark database to a small size, taking into account the length of the compressor itself. This methodology therefore includes an Occam principle as well as a solution to the problem of demarcation. Because of the fundamental difficulty of lossless compression, this type of research must be empirical in nature: compression can only be achieved by discovering and characterizing empirical regularities in the data. Because of this, the philosophy provides a way to reformulate fields such as computer vision and computational linguistics as empirical sciences: the former by attempting to compress databases of natural images, the latter by attempting to compress large text databases. The book argues that the rigor and objectivity of the compression principle should set the stage for systematic progress in these fields. The argument is especially strong in the context of computer vision, which is plagued by chronic problems of evaluation. The book also considers the field of machine learning. Here the traditional approach requires that the models proposed to solve learning problems be extremely simple, in order to avoid overfitting. However, the world may contain intrinsically complex phenomena, which would require complex models to understand. The compression philosophy can justify complex models because of the large quantity of data being modeled (if the target database is 100 Gb, it is easy to justify a 10 Mb model). The complex models and abstractions learned on the basis of the raw data (images, language, etc) can then be reused to solve any specific learning problem, such as face recognition or machine translation.
An Integrated Modeling Environment to Study the Co-evolution of Networks, Individual Behavior and Epidemics
Barrett, Christopher (Network Dynamics and Sim Science Lab) | Bisset, Keith (Network Dynamics and Sim Science Lab) | Leidig, Jonathan (Network Dynamics and Sim Science Lab) | Marathe, Achla (Network Dynamics and Sim Science Lab) | Marathe, Madhav V. (Network Dynamics and Sim Science Lab)
We discuss an interaction-based approach to study the coevolution between socio-technical networks, individual behaviors, and contagion processes on these networks. Finally, models of individual behaviors are composed with disease progression models to develop a realistic representation of the complex system in which individual behaviors and the social network adapt to the contagion. These methods are embodied within Simdemics – a general purpose modeling environment to support pandemic planning and response. New advances in network science, machine learning, high performance computing, data mining and behavioral modeling were necessary to develop Simdemics.
2003 AAAI Robot Competition and Exhibition
Maxwell, Bruce A., Smart, William, Jacoff, Adam, Casper, Jennifer, Weiss, Brian, Scholtz, Jean, Yanco, Holly, Micire, Mark, Stroupe, Ashley, Stormont, Dan, Lauwers, Tom
The Twelfth Annual Association for the Advancement of Artificial Intelligence (AAAI) Robot Competition and Exhibition was held in Acapulco, Mexico, in conjunction with the Eighteenth International Joint Conference on Artificial Intelligence. The events included the Robot Host and Urban Search and Rescue competitions, the AAAI Robot Challenge, and the Robot Exhibition. In the Urban Search and Rescue competition, teams attempted to find victims in a simulated disaster area using teleoperated, semiautonomous, and autonomous robots. The AAAI Robot Challenge is a noncompetitive event where the robots attempt to attend the conference by locating the registration booth, registering for the conference, and then giving a talk to an audience.
The AAAI-2002 Robot Exhibition
The AAAI-2002 Robot Exhibition offered robotics researchers a venue for live demonstrations of their current projects. Researchers ranging from undergraduates working on their own to large multilab groups demonstrated robots that performed tasks ranging from improvisational comedy to urban search and rescue. This article describes their entries.
AAAI/RoboCup-2001 Urban Search and Rescue Events
Murphy, Robin, Blitch, John, Casper, Jennifer
The RoboCup Rescue Physical Agent League Competition was held in the summer of 2001 in conjunction with the AAAI Mobile Robot Competition Urban Search and Rescue event, eerily preceding the September 11 World Trade Center (WTC) disaster. Four teams responded to the WTC disaster through the auspices of the Center for Robot-Assisted Search and Rescue (CRASAR), directed by John Blitch. Blitch, through his position as program manager for the Defense Advanced Research Projects Agency (DARPA) Tactical Mobile Robots Program, was a supporter of the competition; he also served as a member of the rules committee and a judge. USF participated by chairing the rules committee, judging, assisting with the logistics, providing commentary, and demonstrating tethered and wireless robots whenever entrants had to skip around during the competition.