JAN 31, 2019 12:57 PM PST

Translating Brain Signals Directly into Speech

WRITTEN BY: Nouran Amin

In Scientific Reports, neuro-engineers developed a system that translates thought into intelligible and recognizable speech through detections of brain activity that ultimately reconstruct the words a person hears with unprecedented clarity. The system can help people who cannot speak, those living with amyotrophic lateral sclerosis (ALS), or individuals recovering from stroke.

Concept illustration (stock image) via Science Daily  Credit: © adragan / Fotolia

"Our voices help connect us to our friends, family and the world around us, which is why losing the power of one's voice due to injury or disease is so devastating," says Nima Mesgarani, PhD, the paper's senior author and a principal investigator at Columbia University's Mortimer B. Zuckerman Mind Brain Behavior Institute. "With today's study, we have a potential way to restore that power. We've shown that, with the right technology, these people's thoughts could be decoded and understood by any listener."

Previous research has confirmed that when someone speaks or even thinking of saying something—patterns of brain activity begin to surface. Early efforts of the study focused on decoding brain signals through the utilization of computer models that analyzed spectrograms (the visual representations of sound frequencies). However, this approach has failed and so the study shifted to the use of a ‘vocoder’ which is a computer algorithm capable of synthesizing speech after being trained on voice recordings of human conversations

"This is the same technology used by Amazon Echo and Apple Siri to give verbal responses to our questions," said Dr. Mesgarani, who is also an associate professor of electrical engineering at Columbia's Fu Foundation School of Engineering and Applied Science. "Working with Dr. Ashesh Dinesh Mehta (co-author of the study), we asked epilepsy patients already undergoing brain surgery to listen to sentences spoken by different people, while we measured patterns of brain activity," said Dr. Mesgarani. "These neural patterns trained the vocoder."

Source: The Zuckerman Institute at Columbia University

About the Author
  • Nouran enjoys writing on various topics including science & medicine, global health, and conservation biology. She hopes through her writing she can make science more engaging and communicable to the general public.
You May Also Like
FEB 13, 2019
Space & Astronomy
FEB 13, 2019
NASA Officially Terminates the Opportunity Rover Mission
NASA’s Opportunity rover was the interplanetary version of the little engine that could; unfortunately, it couldn’t. The American space agency...
FEB 18, 2019
Space & Astronomy
FEB 18, 2019
Proposed SPHEREx Mission Receives NASA's Blessing, Launching by 2023
The universe is a gigantic place, and while there’s still much we don’t know about it, that hasn’t stopped NASA from making strides in ex...
MAR 12, 2019
Space & Astronomy
MAR 12, 2019
Expedition 59-60: One of the Last Times NASA Will Use a Soyuz Rocket?
A Russian Soyuz MS-12 spacecraft is expected to launch from the Baikonur Cosmodrome on Thursday, March 14th, ferrying Expedition 59-60 to the International...
MAR 19, 2019
Genetics & Genomics
MAR 19, 2019
What is Hachimoji DNA?
  It’s the first thing you learn in biology.  There are four nucleotides which are adenine, cytosine, guanine, and thymine and they make up...
MAR 27, 2019
Technology
MAR 27, 2019
Ever Wonder How Foldable Smartphone Screens Work?
If you have a smartphone in your pocket right now, then we’re willing to bet that it has either a flat or slightly-curved display. But fast-forward t...
APR 16, 2019
Space & Astronomy
APR 16, 2019
TESS Distinguishes its First Earth-Sized Exoplanet Since Launch
It’s been almost exactly one year since a SpaceX Falcon 9 rocket ferried NASA’s Transiting Exoplanet Survey Satellite (TESS) into space to pick...
Loading Comments...