The research described in this paper is motivated by the development of applications for the behaviour analysis of handwriting and sketch input. Our goal is to provide other researchers with a reproducible, categorised set of features that can be used for behaviour characterisation in different scenarios. We use the term feature to describe properties of strokes and gestures which can be calculated based on the raw sensor input from capture devices, such as digital pens or tablets. In this paper, a large number of features known from the literature are presented and categorised into different subsets.
With advancements in technology, smartphones can already serve as memory aids. Electronic calendars are of great use in time-based memory tasks. In this project, we enter the mixed reality realm for helping dementia patients. Dementia is a general term for a decline in mental ability severe enough to interfere with daily life. Memory loss is an example. Here, mixed reality refers to the merging of real and virtual worlds to produce new episodic memory visualisations where physical and digital objects co-exist and interact in real-time. Cognitive models are approximations of a patient's mental abilities and limitations involving conscious mental activities (such as thinking, understanding, learning, and remembering). External representations of episodic memory help patients and caregivers coordinate their actions with one another. We advocate distributed cognition, which involves the coordination between individuals, artefacts and the environment, in four main implementations of artificial intelligence technology in the Kognit storyboard: (1) speech dialogue and episodic memory retrieval; (2) monitoring medication management and tracking an elder's behaviour (e.g., drinking water); (3) eye tracking and modelling cognitive abilities; and (4) serious game development towards active memory training. We discuss the storyboard, use cases and usage scenarios, and some implementation details of cognitive models and mixed reality hardware for the patient. The purpose of future studies is to determine the extent to which cognitive enhancement technology can be used to decrease caregiver burden.
Sonntag, Daniel (German Research Center for AI) | Weber, Markus (German Research Center for AI) | Cavallaro, Alexander (Imaging Science Institute Erlangen) | Hammon, Matthias (Imaging Science Institute Erlangen)
Future radiology practices assume that the radiology reports should be uniform, comprehensive, and easily managed. This means that reports must be readable to humans and machines alike. In order to improve reporting practices in breast imaging, we allow the radiologist to write structured reports with a special pen on paper with an invisible dot pattern. In this way, we provide a knowledge acquisition system for printed mammography patient forms for the combined work with printed and digital documents. In this domain, printed documents cannot be easily replaced by computer systems because they contain free-form sketches and textual annotations, and the acceptance of traditional PC reporting tools is rather low among the doctors. This is due to the fact that current electronic reporting systems significantly add to the amount of time it takes to complete the reports. We describe our real-time digital paper application and focus on the use case study of our deployed application. We think that our results motivate the design and implementation of intuitive pen-based user interfaces for the medical reporting process and similar knowledge work domains. Our system imposes only minimal overhead on traditional form-filling processes and provides for a direct, ontology-based structuring of the user input for semantic search and retrieval applications, as well as other applied artificial intelligence scenarios which involve manual form-based data acquisition.
It captures and analyzes high-precision information about both outcome and process, opening up the possibility of detecting subtle cognitive impairment even when test results appear superficially normal. We describe the design and development of the test, document the role of AI in its capabilities, and report on its use over the past seven years. We outline its potential implications for earlier detection and treatment of neurological disorders. We set the work in the larger context of the THink project, which is exploring multiple approaches to determining cognitive status through the detection and analysis of subtle behaviors. Neurocognitive testing is the overall term for the efforts to assess the performance of our mental capabilities, including for example, memory, attention, problem solving, language and verbal fluency, cognitive processing speed, and others.
Davis, Randall (Massachusetts Institute of Technology) | Libon, David (Drexel University College of Medicine) | Au, Roda (Boston University School of Medicine) | Pitman, David (Kytheram) | Penney, Dana (Lahey Hospital and Medical Center)
The digital clock drawing test is a fielded application that provides a major advance over existing neuropsychological testing technology. It captures and analyzes high precision information about both outcome and process, opening up the possibility of detecting subtle cognitive impairment even when test results appear superficially normal. We describe the design and development of the test, document the role of AI in its capabilities, and report on its use over the past seven years. We outline its potential implications for earlier detection and treatment of neurological disorders. We set the work in the larger context of the THink project, which is exploring multiple approaches to determining cognitive status through the detection and analysis of subtle behaviors.