Augmentative and Alternative Communication Interface with a Context-based, Predictive Language Model

Author/Creator ORCID




Computer Science and Electrical Engineering


Computer Science

Citation of Original Publication


Distribution Rights granted to UMBC by the author.
Access limited to the UMBC community. Item may possibly be obtained via Interlibrary Loan thorugh a local library, pending author/copyright holder's permission.
This item is likely protected under Title 17 of the U.S. Copyright Law. Unless on a Creative Commons license, for uses protected by Copyright Law, contact the copyright holder or the author.



Communication is ubiquitous in the world. The need to describe one's needs and desires is a necessity in all aspects of life. When a person loses the ability to physically speak, and the motor skills to activate speech systems that require physical input, there are limited alternatives for communication. Eye-gaze tracking offers unique opportunities to support communication; however, many of the current eye-gaze tracking communication programs are unintuitive, inaccurate, and outdated. Many of the existing communication programs employ a visual keyboard that the user can use to visually type out the words and sentences they wish to speak. In this paper, I investigate an alternative to the standard virtual keyboard methods, developing a communicative flow that is intuitive and easy to use without speech. I create a simple application for participants to construct messages. I introduce a language model concept that gathers information about the environment and using language groundings, determines the salience of items within a scene. This, in turn, allows the language model to make context sensitive text prediction to better assist a user in communicating quickly and effectively. Using a gaze tracking device as input to the interface, I then perform a user study to evaluate the efficacy and intuitiveness of the communication device. I found that when a user is presented with context based portions of speech, they can communicate easier and quicker than with a generic language model.