Interfaces

Intelligent Communication with Computers

"The use of techniques from the field of autonomous agents provides a new complementary style of human-computer interaction, where the computer becomes an intelligent, active and personalized collaborator. Interface agents are computer programs that employ Artificial Intelligence methods to provide active assistance to a user of a particular computer application. The metaphor used is that of a personal assistant who is collaborating with the user in the same work environment. The assistant becomes gradually more effective as it learns the user's interests, habits and preferences."

- FLUIDS (Intelligent User Interfaces - New Achievements in Research)

Even the most sophisticated and powerful system will be next to useless without an effective user interface. If struggling with the operation of a video player is not your idea of fun, you can well understand why a part of the artificial intelligence community is devoted to scientifically developing interfaces that are easy and pleasant to use, and that form an effective, and possibly seamless, link between the intelligent machine(s) and the human user(s).

Contents

Vertical Tabs

Good Starting Places

Publication

Introduction to the Special Issue on Intelligent User Interfaces

"Recent years have witnessed significant progress in intelligent user interfaces. Emerging from the intersection of AI and human-computer interaction, research on intelligent user interfaces is experiencing a renaissance, both in the overall level of activity and in raw research achievements. Research on intelligent user interfaces exploits developments in a broad range of foundational AI work, ranging from knowledge representation and computational linguistics to planning and vision. Because intelligent user interfaces are designed to facilitate problem-solving activities where reasoning is shared between users and the machine, they are currently transitioning from the laboratory to applications in the workplace, home, and classroom."

Source

By Lester, James, 2011
overview
Link

An Intelligent Personal Assistant for Task and Time Management

By Karen Myers, Pauline Berry, Jim Blythe, Ken Conley, Melinda Gervasio, Deborah L. McGuinness, David Morley, Avi Pfeffer, Martha Pollack, Milind Tambe. AI Magazine, Vol 28, No 2, pp 47-61. "We describe an intelligent personal assistant that has been developed to aid a busy knowledge worker in managing time commitments and performing tasks. The design of the system was motivated by the complementary objectives of (1) relieving the user of routine tasks, thus allowing her to focus on tasks that critically require human problem-solving skills, and (2) intervening in situations where cognitive overload leads to oversights or mistakes by the user. The system draws on a diverse set of AI technologies that are linked within a Belief-Desire-Intention (BDI) agent system. Although the system provides a number of automated functions, the overall framework is highly user centric in its support for human needs, responsiveness to human inputs, and adaptivity to user working style and preferences."

Source

2007
Publication

Mixed-Initiative Assistants

"Mixed-initiative assistants are agents that interact seamlessly with humans to extend their problem-solving capabilities or provide new capabilities. Developing such agents requires the synergistic integration of many areas of AI, including knowledge representation, problem solving and planning, knowledge acquisition and learning, multiagent systems, discourse theory, and human-computer interaction. This paper introduces seven aspects of mixed-initiative reasoning (task, control, awareness, communication, personalization, architecture, and evaluation) and discusses them in the context of several state-of-the-art mixed-initiative assistants. The goal is to provide a framework for understanding and comparing existing mixed-initiative assistants and for developing general design principles and methods."

Contents:

By Gheorghe Tecuci, Mihai Boicu, and Michael T. Cox., 2007
Publication

Reflections on Challenges to the Goal of Invisible Computing

"We know that computers are complex beasts in their own right, but for all of their internal complexity computers are just as complicated in their embedding in the outside world, even though the complexity of this embedding is largely invisible to the people who design computers, and to people who make a living promoting their use. And it is possible that computers [fn] might have the power to change us even when we engage with them unconsciously, as when we relate to a tool through the performance of a skill like driving or typing."


[Department of Philosophy of Technololgy,Institute for Philosophy, Dresden University of Technology, Germany. ]

By Arun Kumar, 2005
Link

Persistent Assistants: Living and Working with AI

Daniel Shapiro, Pauline Berry, John Gersh,and Nathan Schurr. (eds.)  Technical Report SS-05-05. American Association for Artificial Intelligence, Menlo Park, California.

Source

2005
Publication

Intelligent Human-Computer Interfaces

 "This issue is devoted to intelligent human-computer interfaces -- interfaces that raise the computer to the human level, rather than requiring that the human adapt to the computer. An interface may be intelligent because it can communicate using human language, or because it performs intelligent functions, or because it adapts to a specific task and user. In all these cases, an intelligent interface makes interaction with the computer easier, more intuitive, and more flexible. The area of intelligent interfaces is just one facet of the broad area of human-computer interaction. Other important areas include graphical user interface design, usability, human factors, visualization, immersive environments, and intelligent tutoring systems."

The five articles in this issue are: Spoken Language Interfaces Gaining Acceptance as Technology Matures; Command Post of the Future Interactive Visualization to Increase Speed and Quality of Decision Making; DARPA Communicator Program Tackles Conversational Interface Challenges; IMPs Enhance Virtual Collaboration Environments; X-Technologies Ease Customization of Web Interfaces.'''

By Lynette Hirschman, Guest Editor, 1999

General Readings

Publication

IBM Predicts: Cognitive Computers That Feel And Smell, Within The Next Five Years

At the end of each year, IBM releases its "5 in 5" -- five technology predictions that IBM researchers foresee coming to fruition within the coming five years. These predictions are based on everything from emerging market trends to cultural and social behaviors to actual technologies IBM has incubating in its many labs.

Source

Dec 17 2012
taste, touch
AI in the News

Sentry System Combines a Human Brain with Computer Vision

Combining human abilities with those of machines could be useful in many different settings. Sentry duty is a tough assignment.

Source

Nov 27 2012
DARPA
AI in the News

Robots get a feel for the world: Touch more sensitve than a human's

... with the right sensors, actuators and software, robots can be given the sense of feel -- or at least the ability to identify different materials by touch. Researchers at the University of Southern California's Viterbi School of Engineering published a study June 18 in Frontiers in Neurorobotics showing that a specially designed robot can outperform humans in identifying a wide range of natural materials according to their textures, paving the way for advancements in prostheses, personal assistive robots and consumer product testing.

Source

Jun 18 2012
haptic, touch
AI in the News

Software Translates Your Voice into Another Language

Researchers at Microsoft have made software that can learn the sound of your voice, and then use it to speak a language that you don't. In a demonstration at Microsoft's Redmond, Washington, campus on Tuesday, Microsoft research scientist Frank Soong showed how his software could read out text in Spanish using the voice of his boss, Rick Rashid, who leads Microsoft's research efforts. Hear Rick Rashid's voice in his native language and then translated into several other languages: In English, a synthetic version of Mundie's voice welcomed the audience to an open day held by Microsoft Research, concluding, "With the help of this system, now I can speak Mandarin. " Individual sounds used by the first model to build up words using a person's voice in his or her own language are carefully tweaked to give the new text-to-speech model a full ability to sound out phrases in the second language.

Source

Mar 9 2012, By Simonite, Tom
microsoft, speech generation
Link

HCI Bibliography Project

Provided by the HCI Bibliography Project. As of 2012-08-15, the HCI Bibliography has over 71,000 entries. The current search service has been running since 2006-12-01. Includes the HCI Sites page: "a collection of resources on Human-Computer Interaction. It replaced Keith Instone's HCI Virtual Library (originally the HCI Launch Pad)."

Source

2012
AI in the News

Controling robotic arms is child's play

researchers will be presenting an industrial robotic arm with six joints, at the end of which is a catcher. Visitors can control the arm using a hand-held input device: When they move the hand holding the device, the robot emulates their movement.

"The input device contains various movement sensors, also called inertial sensors," says Bernhard Kleiner of the Fraunhofer Institute for Manufacturing Engineering and Automation IPA in Stuttgart, who leads the project. ... "We have developed special algorithms that fuse the data of individual sensors and identify a pattern of movement. That means we can detect movements in free space," summarizes Kleiner.

Source

May 15 2011

Organizations

Link

HCI Research at Stanford

Explore the amazing diversity of contexts in which the science of HCI is being applied.  Links to many projects and publcations from Stanford.

Source

2012
Link

Human Computer Interaction Institute

Carnegie Mellon University. Among the many, and diverse, research projects that you'll find on this page are "Demonstrational Interfaces... the user gives an example of how the system should operate, and the system automatically generalizes from the example to produce a parameterized procedure." and "GM/CMU Project: Driver-Vehicle Interface---The goal is to improve security and driver-vehicle interfaces, by building a car that can analyze the driver's intention and watch the driver's physical and mental status for any impairments or information overload. The application combines a smart car environment and monitoring of driver state, with a wide range of input-output modalities."

Source

2012
CMU, collection, natural language, projects, web
Link

Human Media Lab. Queen's University

 "HML researchers at Queen's University's School of Computing are addressing the problem of the barrage of messages people receive from large numbers of digital appliances. Their Attentive User Interface (AUI) is a new paradigm for interacting with groups of computers that moves beyond the traditional desktop interface. Current computers are generally designed to act in isolation, without considering what the user is doing before producing distracting interruptions. As a result, todays user has trouble keeping up with volumes of e-mail, instant messages, phone calls and appointment notifications. 'Todays digital lifestyle has the unfortunate side effect of bombarding people with messages from many devices all the time, regardless of whether theyre willing, or able to respond,' says HML director Dr. Roel Vertegaal. 'Like spam [unsolicited e-mail], this problem needs to be addressed.' The HML team is designing devices that determine the level of user attention and the importance of each message relative to what the user is doing. Then the computer decides whether to 'take a turn' to deliver the message."---from Researchers invent computers that pay attention to users. Queen's News Centre (April 1, 2003).

Source

2003
Link

Intelligent Information Laboratory (Northwestern Univ.)

Visit the project page at the Intelligent Information Laboratory (InfoLab) at Northwestern University and read their entire collection of related papers.

Source

Link

Agent-based Intelligent Reactive Environments (AIRE)

Agent-based Intelligent Reactive Environments---a Research Group at the MIT Artificial Intelligence Laboratory. "aire is dedicated to examining how to design pervasive computing systems and applications for people. To study this, aire designs and constructs Intelligent Environments (IEs), which are spaces augmented with basic perceptual sensing, speech recognition, and distributed agent logic."

Source

MIT

Educational Resources

Link

HCI Bibliography

Provided by the HCI Bibliography Project. Over 58,000 records (as of spring 2010) including a definition of HCI, the history of HCI, recommended readings, conferences, and much more.

Source

2010
bibliography, collection, resources
Link

Curricula for Human-Computer Interaction

ACM Special Interest Group on Computer-Human Interaction Curriculum Development Group

 

 

Source

2009
courses, resources, syllabi

Hardware and Software

AI in the News

Assistive Technology Computer Products for Low Vision Designed by Ai Squared ...

The products from Ai Squared are available in 20 languages and are sold in 45 countries throughout the world. Their state-of-the-art products include ZoomText, ZoomText USB, the ZoomText Large-Print Keyboard and ZoomText Express. They introduced their flagship product, ZoomText, in 1988 and as the popularity of this product grew, the company shifted its focus to the exclusive development and marketing of software for the visually impaired. Their state-of-the-art products include ZoomText, ZoomText USB, the ZoomText Large-Print Keyboard and ZoomText Express.

Source

May 5 2012
product announcement
AI in the News

Watch This Robot Control a Person's Arm Using Electrodes

When this robot needs a hand, it borrows yours. In an experiment that opens a new chapter in human-machine interaction, a French research team has demonstrated how a robot can control both its own arm and a person s arm to manipulate objects in a collaborative manner. From a medical point of view, you might want to encourage the person to move more, and that s when the robot can help, by moving the person s arm to reach and hold the glass. The audience at the IEEE International Conference on Intelligent Robots and Systems (IROS), where the researchers presented their results in September, let out a nervous gasp upon seeing the video of the experiment: In the study, Adorno and his colleagues, Ant nio Padilha Lanari B and Philippe Fraisse, attached four electrodes to the arms of five healthy volunteers test subjects, who were also blindfolded, to avoid visual feedback.

Source

Nov 15 2011

Journals and Proceedings

Link

Intelligent User Interfaces (IUI)

Intelligent User Interfaces (IUI) is the premier conference for reporting on the study of user interfaces with intelligent devices.

  • Intelligent interaction with devices

• Intelligent interactions with handheld devices such as mobile phones, music and video players, GPS, cameras, electronic books, etc. • Sensor- and actuator systems for user interfaces • Location- and context aware information systems • Tangible interaction with smart artifacts • Ubiquitous displays environments • Smart environments Novel, intelligent interaction systems • Modeling and prediction of user behavior • Affective, social and aesthetic interfaces • Natural user interfaces including tangible computing and body computing • User-adaptivity in interactive systems • Personalization and recommender systems • Planning and plan recognition

  • IUI design

• Knowledge-based approaches to user interface design and generation • Proactive and agent-based paradigms for user interaction • Example-based and demonstration-based interfaces • Smart use of sensing technologies for IUI Design User studies • User studies concerning intelligent interfaces • Evaluation methods and evaluations of implemented intelligent user interfaces • Smart technologies for remote usability testing and experience sampling

  • Processing of human-generated input

• Recognition and interpretation of user input (face, body, speech, physiology, text) • Analysis of psychological user states, such as attention and affect • Analysis of conversational cues, such as grounding and turn taking • Intelligent sensing platforms • Synchronization and fusion of Multimodal Input

  • Generation presentation of system output

• Smart visualization tools • Intelligent authoring systems • Context-aware systems for the generation and presentation of situation-specific output (e.g., on mobile phones, portable tablet devices, wall-size displays, multi-touch screens, meeting accessibility criteria) • Synthesis of multimodal behavior for virtual characters and social robots Intelligent help assistants for complex tasks • Support for collaboration in multiuser environments • Intelligent information and knowledge management

  • Novel trends, innovative solutions and applications

• Novel sensing technologies for games and entertainment • Automotive user interfaces • Novel interaction methods for touch phones and multi touch surfaces

 

Source

2011

News

AI in the News

With Swype, Nuance Begins Quest To Process What We Say -- Everywhere

Last night Nuance released Swype on Google Play, marking the swishy keyboard app's debut on the store after being primarily embedded into smartphones or available through the Android marketplace. It also represented a broader goal of Nuance as a speech recognition company: creating personal dictionaries that better understand what we say and type to a machine, be it a smartphone, car or laptop.

Source

Apr 25 2013
AI in the News

Memory Implants

Theodore Berger, a biomedical engineer and neuroscientist at the University of Southern California in Los Angeles, envisions a day in the not too distant future when a patient with severe memory loss can get help from an electronic implant. In people whose brains have suffered damage from Alzheimer's, stroke, or injury, disrupted neuronal networks often prevent long-term memories from forming.

Source

Apr 23 2013
AI in the News

Samsung Galaxy S4 Release Nears As Company Highlights The 'Hidden Innovation' In Its Newest Smartphone

Samsung's highly anticipated Galaxy S4 boasts a host of exciting new features including Smart Scroll, Smart Pause, Sound & Shot and Group Play; however, Samsung is now taking the time to highlight the "hidden innovation" the smartphone also possesses.

Source

Apr 10 2013
AI in the News

Kinect-Powered Depression Detector is Amazing and Creepy

People have been baring their souls to soulless machines ever since ELIZA successfully impersonated a Rogerian therapist using nothing but a text display in the 1960s. Still, any value gotten out of these interactions was totally projected by the human user; the dumb code couldn't actually interpret your feelings.

Source

Apr 1 2013
AI in the News

Beyond Voice Recognition: It's The Age Of Intelligent Systems

For many people, 2012 was the year when it became "normal" to use your voice to control your phone, car, computer and even your TV.

Source

Jan 11 2013
AI in the News

PC Makers Bet on Gaze, Gesture, Voice, and Touch

Products that could make it common to control a computer, TV, or something else using eye gaze, gesture, voice, and even facial expression were launched at the Consumer Electronics Show in Las Vegas this week. The technology promises to make computers and other devices easier to use, let devices do new things, and perhaps boost the prospects of companies reliant on PC sales.

Source

Jan 10 2013

Videos

Video

IBM Demonstrates Speech-to-Sign Language Translation System

"Say It Sign It" avatar translates the spoken word into British Sign Language.

Source

1 min, 2008,
video
Video

CALO Workflow Recognition and Proactive Assistance

SRI's CALO agent helps users manage their tasks and time. Using advanced activity recognition algorithms based on logical probabilistic models, CALO observes and tracks user desktop actions and offers contextually appropriate assistance.

1 min, 2008, AAAI
Video

Linking Brains, Computers

Source

3 min, 2008
video
Video

The Lumiere Project: Intelligent User Interface

1995 video on the Lumiere project at Microsoft Research showing a prototype intelligent assistance facility that observes a stream of evidence and that infers goals and needs. Bayesian and decision-theoretic models are employed to make inferences about goals, including the fusing of evidence provided by words from users' queries and from user actions.

Source

2007, Microsoft
video

Podcasts

Podcast

Machine Men

We hear passages from the novel Machine Man by Max Barry, about a man who slowly transforms himself, body part by body part, into a machine. And then we'll meet a real life amputee, an MIT professor named Hugh Herr, who is building mechanical body parts for real.

Aug 22 2012, 14 min, The Truth

Classic Articles & Books

Publication

Natural language interfaces

Source

By Perrault, Ray and Barbara Grosz, 1988
classic

Vertical Tabs