A faculty team in the College of Education and and Health Professions is leading a nearly $4.8 million project — in collaboration with Florida State, Auburn, Penn State and Temple universities — to develop new artificial intelligence-enhanced technologies to help people with significant speech and language limitations communicate more easily and precisely with the world around them.

The overarching goal of the five-year Rehabilitation Engineering Research Center on Augmentative and Alternative Communication grant, funded by the Department of Health and Human Services, is to improve augmentative and alternative communication devices, or AAC devices for short, so the people with developmental disabilities who use them are freer to fulfill their dreams and priorities.

"If someone has access to tools that really meet their needs to build their communication and language, they're more likely to have meaningful employment, be successful in school, have friends and be involved in the community," said Christine Holyfield, an associate professor of communication sciences and disorders and the principal investigator on the grant.

A key approach is to actively involve AAC users in the process of improving the devices.

"People who use AAC devices will have prominent roles throughout the life of this grant," Holyfield said, noting that's rare in research. "The technology will be more useful to more people if we involve those who will use it through every step of the process. This funding allows us to recruit more participants."

Holyfield is joined by two faculty members in the College of Education and Health Professions' special education program, Elizabeth Lorah and Sheida Raley. Communication sciences and disorders faculty from Florida State, Auburn and Penn State are conducting research and development as part of the grant, while engineering faculty from Temple will contribute their computer science expertise.

Holyfield said the current AAC technology — which allows users to select words or pictures on a screen to generate corresponding speech — is life-changing for the people with developmental disabilities who require it, but there's considerable potential for improvement.

The grant supports at least six research and development projects, and they all have an AI component. Currently, AAC users must invest a lot of time learning how to operate their devices before they can use them effectively.

"With AI, we hope the devices can learn about users rather than just the other way around," Holyfield said. "With smarter AAC technology, we can integrate other forms of communication beyond device selections, such as vocalizations or body language. We can embed communication within the environment, and we can make devices responsive to that environment. We can also provide more effective opportunities for learning to communicate, learning language and learning literacy. Of course, as we all experience through AI-enabled communication today, we can also make communication more efficient. This is particularly meaningful for AAC users as they are a group for whom communication can take so long that conversations often move on before they can contribute."

Holyfield said right now, many AAC users are also reliant on others to provide them with communication opportunities. She envisions a future where AI-enabled AAC devices can better support AAC users in determining what and when they communicate. For example, a child with Down syndrome could see an insect of interest on a class nature walk, and by taking a photo with her AAC device, could see automated related communication options.

With AI able to better "understand" an individual user, conversations could be faster and feel more personal. As noted in a recent journal paper co-authored by Holyfield, technology might detect AAC users' gestures or facial expressions and modulate speech output accordingly to inform partners of the individual's communicative intent, supporting greater partner responsivity, a key component in language development.

In addition to novel AAC features and other supports made commercially available in collaboration with manufacturers, the expected products from the project include freely available prototypes for exploration and further testing; publications, learning modules and other informational materials; a State of the Science conference; and a doctoral training course. 

The grant builds on work Holyfield and Lorah's AAC Lab has done to advance research in Arkansas, which was supported by multiple WE CARE grants from the College of Education and Health Professions. WE CARE stands for Wellness and Education Commitment to Arkansas Excellence.