There are approximately 4 million Americans with severe speech and physical impairments (SSPIs) who must rely on augmentative and alternative communication (AAC) systems to express themselves. People with SSPI are at a significant disadvantage when communicating. AAC systems place heavy cognitive and physical demands on users, resulting in very slow communication rates. The AAC user spends so much time and energy constructing messages, there is often little time for typical turn taking sequences or sustained conversation. Cultural rules for successful interactions are violated. The communication partners are disadvantaged as well, often waiting for minutes with no means for reciprocal engagement before a single message is generated. Currently, the communication partners have no clear mechanism to help the AAC user formulate messages, even if they have shared knowledge about the vocabulary. Is there a way for us to capitalize on the physical skills, language skills and shared knowledge of the communication partners to enhance the user's message production and communication performance? We propose to develop an innovative technology-based system for AAC conversation that is grounded in the success and efficiency of dialogue co-construction and word prediction. We combine dynamic contextual word/phrase prediction from non-disabled communication partners with a sophisticated bigram-based language model residing in the AAC system to increase the speed and informativeness of message production. In Phase 1, we will develop two Android apps (Construct-AAC and Constructor) that communicate via Bluetooth, one for the AAC user and one for the non-disabled partner (called the co-constructor). The AAC user spells out a message with Construct-AAC. The second app, Constructor, displays a copy of the AAC user's message as it is being created and enables the co-constructor to suggest words or phrases in real time by either typing out words, or by choosing words from a prediction list. The word and phrase predictions, along with other lexica stored in the device, are prioritized by the language model within the AAC user's app, and presented in standard word prediction lists to the AAC user. The AAC user maintains independence to choose or ignore the co-construction suggestions during message generation. This written interaction mimics the co-construction behavior observed in speaking conversations and offers the two partners a common written milieu for reciprocal, face-to-face, verbal interaction. Feasibility of this new AAC system will be tested using a single case ABAB withdrawal design with ten dyads of non-disabled participants and five dyads of AAC users and their non-disabled co-construction partners. Participants are shown the Boston Cookie Theft Picture and instructed to produce a language sample by describing the scene. In condition a, participants use only Construct-AAC to describe the picture. In the experimental condition B, the co-construction partner contributes to the user's picture description with the Constructor app. We predict that having access to contextual predictions will improve speed and informativeness of AAC output. This, in turn, will positively influence conversational performance. In Phase 2, we will optimize the system and examine the effect of enhanced AAC output on conversation.
The proposed research may enable people with severe speech and physical impairments to communicate more effectively, enabling them to participate more fully in their health care, leisure activities and decision making.
