Mind-Reading Program Can Now Translate Your Thoughts Into Text

by Monica Joshi

Technology has gotten so advanced that speaking into your phone can allow for text to come out on the screen. Technology can also reconstruct video based on a person's thoughts and even anticipate your moves while you drive. A new program, a brain-to-text system, can now translate brain activity into written words.

In a recent study, published in Frontiers in Neuroscience, seven patients had electrode sheets placed on their brains that collected neural data while they read passages aloud from the Gettysburg Address, JFK’s inaugural speech, and Humpty Dumpty.

While each patient spoke, a computer algorithm learned to associate speech sounds, such as "foh", "net", and "ik", with different firing patterns in the brain cells.

Later, it learned to read the brain cells well enough that it could guess which sound they were producing with up to 75 percent accuracy. However, the program doesn't need 100 percent accuracy to put those sounds together into the word "phonetic".

As our speech only really takes certain forms, the system’s algorithm can correct for these errors. Kind of like autocorrect.

Peter Brunner, one of the authors of the study, said that:

“Siri wouldn’t be more accurate than 50 or 70 percent. Because it knows what the potential options are that you choose, or the typical sentences that you say, it can actually utilize this information to get the right choice.”

It seems that data directly recorded from the brain will be far more accurate, as picking up neural activity from the scalp only gives a “blurred version” of what is happening in the brain. Brunner likened the latter method to flying 1000 feet above a baseball stadium and only being able to vaguely recognize that people are cheering. However, he added that they wouldn't know the specifics of what the people’s faces look like.

In this case, the patients were already undergoing an epilepsy procedure where the skull is popped open and an electrode grid is placed on the brain to map areas where neurons are misfiring. Hence, there were few such viable candidates to study. The resourceful team of researchers from the National Center for Adaptive Neurotechnologies and the State University of New York at Albany used this time to conduct their own research without probing too much into healthy brains. This, however, limited the study based on each patient’s individualized epilepsy treatment, such as where the electrodes were placed on the brain.

While the study might be exciting and might even create new possibilities for further research, it would be a hard bite. Because every person’s brain is so unique, and the neural activity must be picked up directly from the brain, it would be difficult to create a general brain-to-text device for the average consumer.

This technology, however, does have a lot of potential to be used for people who suffer from neurological diseases, such as ALS, who lose the ability to move and speak. Instead of using an external device - like the one that Stephen Hawking needs to - to pick out words on a screen for a computer to read, the computer would simply speak his mind. He would no longer have to preemptively queue up his jokes, but would be able to joke straight from the brain, if you will.