Coming with the ever growing computational power of mobile devices, mobile visual search have undergone an evolution in techniques and applications. A significant trend is low bit rate visual search, where compact visual descriptors are extracted directly over a mobile and delivered as queries rather than raw images to reduce the query transmission latency. In this article, we introduce our work on low bit rate mobile landmark search, in which a compact yet discriminative landmark image descriptor is extracted by using location context such as GPS, crowd-sourced hotspot WLAN, and cell tower locations. The compactness originates from the bag-of-words image representation, with an offline learning from geotagged photos from online photo sharing websites including Flickr and Panoramio. The learning process involves segmenting the landmark photo collection by discrete geographical regions using Gaussian mixture model, and then boosting a ranking sensitive vocabulary within each region, with an “entropy” based descriptor compactness feedback to refine both phases iteratively. In online search, when entering a geographical region, the codebook in a mobile device are downstream adapted to generate extremely compact descriptors with promising discriminative ability. We have deployed landmark search apps to both HTC and iPhone mobile phones, working over the database of million scale images in typical areas like Beijing, New York, and Barcelona, and others. Our descriptor outperforms alternative compact descriptors (Chen et al. 2009; Chen et al., 2010; Chandrasekhar et al. 2009a; Chandrasekhar et al. 2009b) with significant margins. Beyond landmark search, this article will summarize the MPEG standarization progress of compact descriptor for visual search (CDVS) (Yuri et al. 2010; Yuri et al. 2011) towards application interoperability.

Since people spend a majority of their time indoors, indoor air quality (IAQ) can have a significant impact on human health, safety, productivity, and comfort. Due to the diversity and dynamics of people's indoor activities, it is important to monitor IAQ for each individual. Most existing air quality sensing systems are stationary or focus on outdoor air quality. In contrast, we propose MAQS, a user-centric mobile sensing system for IAQ monitoring. MAQS users carry portable, indoor location tracking and IAQ sensing devices that provide personalized IAQ information in real time. To improve accuracy and energy efficiency, MAQS incorporates three novel techniques: (1) an accurate temporal n-gram augmented Bayesian room localization method that requires few Wi-Fi fingerprints; (2) an air exchange rate based IAQ sensing method, which measures general IAQ using only CO$_2$ sensors; and (3) a zone-based proximity detection method for collaborative sensing, which saves energy and enables data sharing among users. MAQS has been deployed and evaluated via a real-world user study. This evaluation demonstrates that MAQS supports accurate personalized IAQ monitoring and quantitative analysis with high energy efficiency. We also found that study participants frequently experienced poor IAQ.

We will see more and more applications of AI on the mobile devices. This special issue of AI Magazine is devoted to some exemplary works of AI on mobile devices. We include four works that range from mobile activity recognition and air-quality detection to machine translation and image compression. These works were chosen from a variety of sources, including the International Joint Conference on Artificial Intelligence 2011 Special Track on Integrated and Embedded AI Systems, held in Barcelona, Spain, in July 2011.

This paper proposes techniques for preliminary disease detection and personal symptom tracking adopting concepts and methods of web information retrieval. The proposed approaches are inspired by web users’ behavior. People look for information of symptoms from Internet. Therefore, considering information in Web pages, the developed system proposes possible diseases related to one or more queried symptoms. Moreover, these queried symptoms would be recorded in the query log so that the user could utilize these records to trace the history of symptoms, further to manage their own health or provide them to doctors as reference. As ranking detected diseases needs professional knowledge, we instead evaluate relevancy of retrieved sentences containing detected diseases in both strict and lenient metrics. Experimental results support the proposed ranking approach. The techniques described in this paper are also implemented to develop an Android application called “Health Generation”. In this application, the detected disease is further linked to its Wikipedia introduction and the nearby clinics are listed. Users can utilize the GPS function provided by cell phones to plan the route for them. Through the proposed approaches and the application to provide medical information and solutions according to users’ need and further to help users manage their health is the aim of this research.

In this paper we discuss the gap that exists between the caregivers of older adults attempting to age-in-place and sophisticated ”smart-home” systems that can sense the environment and provide assistance when needed. We argue that smart-home systems need to be customizable by end-users, and we present a general-purpose model for cognitive assistive technology that can be adapted to suit many different tasks, users and environments. Al- though we can provide mechanisms for engineers and designers to build and adapt smart-home systems based on this general-purpose model, these mechanisms are not easily understood by or sufficiently user-friendly for actual end users such as older adults and their care- givers. Our goal is therefore to study how to bridge the gap between the end-users and this technology. In this paper, we discuss our work on this problem from both sides: developing technology that is customizable and general-purpose, and studying user’s abilities and needs when it comes to building smart-home systems to help with activities of daily living. We show how a large gap still exists, and propose ideas for how to bridge the gap.

Wandering is a problematic behavior that is common among people with dementia (PwD), and is highly influenced by the elders’ background and by contextual factors specific to the situation. We have developed the Ambient Augmented Memory System (AAMS) to support the caregiver to implement interventions based on providing external memory aids to the PwD. To provide a successful intervention, it is required to use an individualized approach that considers the context of the PwD situation. To reach this end, we extended the AAMS system to include an ontological model to support the context-aware tailoring of interventions for wandering. In this paper, we illustrate the ontology flexibility to personalize the AAMS system to support direct and indirect interventions for wandering through mobile devices.

The Melody Triangle is an interface for the discovery of melodic materials, where the input – positions within a triangle – directly map to information theoretic properties of the output. A model of human expectation and surprise in the perception of music, information dynamics, is used to ‘map out’ a musical generative system’s parameter space. This enables a user to explore the possibilities afforded by a generative algorithm, in this case Markov chains, not by directly selecting parameters, but by specifying the subjective predictability of the output sequence. We describe some of the relevant ideas from information dynamics and how the Melody Triangle is defined in terms of these. We describe its incarnation as a screen based performance tool and compositional aid for the generation of musical textures; the users control at the abstract level of randomness and predictability, and some pilot studies carried out with it. We also briefly outline a multi-user installation, where collabo- ration in a performative setting provides a playful yet informative way to explore expectation and surprise in music, and a forthcoming mobile phone version of the Melody Triangle.

Location-based games are an emerging paradigm fortraining, simulation, entertainment, health and many other domains. In this paper, we consider the role of artificialagents in such games. We also examine how human teams perform when given the same game, playedin both a real environment with mobile devices and alsoin a virtual environment that replicates the real environment.We perform the first direct comparison of real andvirtual instantiations of the same location-based game.We show the similarities and differences in game playand then investigate how adding an advice-giving agentchanges the experience.

Location-based games are an emerging paradigm for training, simulation, entertainment, health and many other domains. In this paper, we consider the role of location-based games as a platform for data collection and analysis of human behavior. We also examine how human teams perform in a disaster scenario when such a scenario is mapped to a game environment conducted as a location-based augmented reality game. We use a pilot experiment to study human behavior between simulated disaster rescue teams and an integrated commander for the purpose of future research into improving exploitation of local tasks versus exploration of assigned objectives by disaster response teams. We show the results of our pilot experiment, analyze the effectiveness of this game as a data collection platform and then investigate how additional experiments may be conducted to formalize this problem further.

With the growing popularity of social networks and collaboration systems, people are increasingly working with or socially connected with each other. Unified messaging system provides a single interface for users to receive and process information from multiple sources. It is highly desirable to design attention management solution that can help users easily navigate and process dozens of unread messages from a unified message system. Moreover, with the proliferation of mobile devices people are now selectively consuming the most important messages on the go between different activities in their daily life. The information overload problem is especially acute for mobile users with small screen to display. In this paper, we present \PAM, an intelligent end-to-end Personalized Attention Management solution that employs analytical techniques that can learn user interests and organize and prioritize incoming messages based on user interests. For a list of unread messages, \PAM generates a concise attention report that allows users to quickly scan the important new messages from his important social connections as well as messages about his most important tasks that the user is involved with. Our solution can also be applied in other applications such as news filtering and alerts on mobile devices. Our evaluation results demonstrate the effectiveness of \PAM.