With the help of a system that reads electrical signals from the speech areas in his brain, a man who has been unable to speak since a stroke has formulated sentences. This was recently reported by researchers working with Edward Chang from the University of California, San Francisco. The procedure has already been used on non-disabled volunteers to reconstruct spoken or made-up sentences. But this first demonstration on a paralyzed person “really addresses the main problem that needed to be solved – getting it to the patients who really need it,” says Christian Herff, a computer scientist at Maastricht University who is not involved in the new Work was involved.
The subject had a stroke more than ten years ago that resulted in anarthria, an inability to control the muscles involved in speaking. Since his limbs are also paralyzed, he communicates by selecting letters on a screen with small movements of his head. This way he forms about five words per minute. In order to enable faster and more natural communication, the neurosurgeon Chang tested an approach with a so-called deep learning algorithm. This interprets patterns of brain activity in the sensorimotor cortex, a brain region that is involved in speaking. The approach has so far been tested in volunteers who have been implanted with electrodes for other reasons, such as monitoring epileptic seizures.
In the new study, Chang’s team temporarily removed part of the participant’s skull and placed a thin plate of electrodes, smaller than a credit card, directly over his sensorimotor cortex. To train a computer algorithm that would associate patterns of brain activity with the onset of speech and certain words, the team first had to determine what the man intended to say and when.
18 words per minute via the brain interface
So the researchers repeatedly presented one word out of 50 on a screen and asked the man to try saying it on call. After training the algorithm on the data from the single word problem, the man tried to read sentences made up of the same group of 50 words, such as “bring me my glasses, please”. To improve the algorithm’s guesswork, the researchers added a component called the natural language model. This uses common word sequences to predict the next word in a sentence with a high probability. With this approach, the system was only wrong about 25 percent of the words in a sentence, report the neurologists in New England Journal of Medicine. That’s “pretty impressive,” says Stephanie Riès-Cornou, a neuroscientist at San Diego State University. The error rate for a random performance would be 92 percent.
Since the brain reorganizes itself over time, it was not clear whether the speech-producing areas would still provide interpretable signals after more than ten years of anarthria, notes Anne-Lise Giraud, neuroscientist at the University of Geneva. The preservation of the signals “is surprising,” she says. Herff says the team took a “giant” step by generating sentences while the man was trying to speak. And not, as is usual in most studies, from previously recorded brain data.
With the new approach, the man could produce sentences at up to 18 words per minute, Chang said. This is roughly comparable to the speed that was achieved with another brain-computer interface in May in Nature described became. This system decoded individual letters from the activity in a brain area that is responsible for planning hand movements while a paralyzed person imagined handwriting. These speeds are still a long way from the 120 to 180 words per minute typical of colloquial English, says Riès-Cornou, but they far exceed what the participant can achieve with his head-controlled device.
The system is not yet suitable for everyday use, notes Chang. Future improvements will be to expand the repertoire of words and make the system wireless so that the user is not tied to a computer the size of a mini fridge.
The original of this article is in the science magazine Science published by the AAAS. German version: cvei.