Brain-to-Text Communication Neuroprosthesis

Abstract

Researchers at the University of California, Davis have developed a Brain-Computer Interface (BCI) technology that enables individuals with paralysis to communicate and control devices through multimodal speech and gesture neural activity decoding.

Full Description

Developed by the University of California, Davis Neuroprosthetics Lab, this advanced BCI system decodes neural activity related to attempted speech into text and other controls (e.g., cursor control, emoji gestures), offering real-time communication and computer interaction for individuals with conditions like ALS or stroke. The system, which includes features such as contextual speech decoding, the ability to learn new words and self-supervise its continuous fine-tuning, has demonstrated a very high degree of accuracy in decoding attempts by a severely dysarthric person, who is now using it independently daily for months on end.

Applications

• Assistive technologies for individuals with speech and motor impairments. 
• Home and professional care settings for patients with ALS, stroke, or similar conditions. 
• Speech therapy and rehabilitation tools. 
• Human-computer interaction research and development.

Features/Benefits

• Enables a very high degree of accuracy in decoding attempted speech into text in real-time. 
• Integrates with text-to-speech tools to vocalize decoded text in the user's own voice. 
• Offers multi-modal functionality, including computer cursor control. 
• Integrates with standard consumer electronics as a keyboard and mouse to enable versatile digital use. 
• Decodes the user’s intended facial expressions, emotional state and/or other gestures to provide additional expressivity during communication. 
• Employs contextual speech decoding to enhance accuracy. Allows users to teach the system new words, improving its versatility and adaptability. 
• Facilitates word-by-word sentence correction for efficient communication and providing new training data for the decoding algorithm. 
• Exploits intrinsic neural error signals to assist output correction and improve the system through self-supervision. 
• Enables caregiver-initiated system initialization for daily, independent use. 
• Addresses limited communication abilities of individuals with severe speech and motor impairments. 
• Reduces dependency on caregivers or professionals for system operation. 
• Addresses challenges in adapting to users' unique linguistic preferences and errors in speech decoding. 
• Eliminates restrictions on the user's ability to independently use computers and other digital devices.

Patent Status

Patent Pending