Researchers at Stanford University have designed an innovative brain-machine interface (BMI). This direct neural interface will allow the user to write words by thinking about how they would have written them in handwritten letters. This new technology could make it easier for people who cannot write because of a disability to express themselves.

Brain-machine interfaces to help people with disabilities in their daily lives

BMIs connect the human brain to external devices such as a computer and are intended to improve the quality of life of people with disabilities. These devices translate brain activity to “express” a user’s wishes. For example, in MCI for communication, a cursor can move across a virtual keyboard so that a person can type words on a computer. Others track eye movement so that the cursor moves correctly.

In a study, several researchers at Stanford University wanted to go further. This is what they explain in an article written by Darrel R. Deo, Paymon Rezai, Leigh R. Hochberg, Allison Okamura, Krishna Vaughn Shenoy and Jamie M. Henderson.

Neural networks to translate thoughts from writing

This MCI relies on an artificial neural network that can translate the individual’s brain signals. The person imagines writing something with a pen, and the model takes into account his or her thoughts to reproduce them on the screen. The main advantage of this device is that the person can continue to look at what they want or think about what they want to write without thinking about the movement of a cursor or forcing themselves to look at a particular place.

Two sensor arrays are used to capture the signals emitted by about 100 neurons. These sensors are placed on the user who only has to imagine writing words on a sheet of paper with a pencil. The handwriting is then automatically converted into text.

The tool was tested by a man whose hand was paralyzed due to a spinal cord injury. In this experiment, the participant wrote about 90 characters per minute with a raw accuracy of 94.1%, a figure that rises to 99% if the device is combined with a universal automatic proofreader. For comparison, the researchers proposed to other able-bodied people of the same age to use their smartphones to write the same words: their typing speed is estimated at 115 characters per minute, which is only 25 more than that proposed by the ICM.

A system trained by machine learning and data augmentation

However, the system has its limitations: the sensors do not target all the neurons directly involved in the control of hand movement. This task would be far too complex given that there are several thousand or even several million neurons dedicated to this task in the human brain. The two sensors monitor about 200 neurons, which according to the researchers, is enough to obtain a reliable interpretation of the brain signals.

The tool was trained using a machine learning algorithm: the research team used a few examples of search signals generated by the human brain when thinking about writing a letter of the alphabet. The researchers then enriched the dataset by performing a data augmentation, which consisted of duplicating the samples while adding subtle interferences and variations to obtain a larger training dataset.

Based on the same principle, the team now wants to design a speech decoder that could allow mute people to synthesize their speech by thinking and interpreting the neural signals usually associated with speech.

Translated from Une interface cerveau-machine permet à des personnes handicapées d’écrire grâce à la pensée