Traditional symbol-based AAC devices impose meta-linguistic and memory demands on individuals with complex communication needs and hinder conversation partners from stimulating symbolic language in meaningful moments. This work presents a prototype application that generates situation-specific communication boards formed by a combination of descriptive, narrative, and semantic related words and phrases inferred automatically from photographs. Through semi-structured interviews with AAC professionals, we investigate how this prototype was used to support communication and language learning in naturalistic school and therapy settings. We find that the immediacy of vocabulary reduces conversation partners' workload, opens up opportunities for AAC stimulation, and facilitates symbolic understanding and sentence construction. We contribute a nuanced understanding of how vocabularies generated automatically from photographs can support individuals with complex communication needs in using and learning symbolic AAC, offering insights into the design of automatic vocabulary generation methods and interfaces to better support various scenarios of use and goals.

For people constrained to picture based communication, the expression of interest in a question answering (QA) or information retrieval (IR)scenario is highly limited. Traditionally, alternative and augmentative communication (AAC) methods (such as gestures and communication boards) are utilised. But only few systems allow users to produce whole utterances or sentences that consist of multiple words; work to generate them automatically is a promising direction in the big data context.In this paper, we provide a dedicated access method for the open-domain QA and IR context. We propose a method for the user to search for additional symbols to be added to the communication board in real-time while using access to big data sources and context based filtering when the desired symbol is missing. The user can select a symbol that is associated with the desired concept and the system searches for images on the Internet - here, in Wikipedia - with the purpose of retrieving an appropriate symbol or picture. Querying for candidates is performed by estimating semantic relatedness between text fragments using explicit semantic analysis (ESA).

For people constrained to picture based communication, the expression of interest in a question answering (QA) or information retrieval (IR)scenario is highly limited. Traditionally, alternative and augmentative communication (AAC) methods (such as gestures and communication boards) are utilised. But only few systems allow users to produce whole utterances or sentences that consist of multiple words; work to generate them automatically is a promising direction in the big data context.In this paper, we provide a dedicated access method for the open-domain QA and IR context. We propose a method for the user to search for additional symbols to be added to the communication board in real-time while using access to big data sources and context based filtering when the desired symbol is missing. The user can select a symbol that is associated with the desired concept and the system searches for images on the Internet - here, in Wikipedia - with the purpose of retrieving an appropriate symbol or picture. Querying for candidates is performed by estimating semantic relatedness between text fragments using explicit semantic analysis (ESA).