JAN 31, 2019 12:57 PM PST

Translating Brain Signals Directly into Speech

WRITTEN BY: Nouran Amin

In Scientific Reports, neuro-engineers developed a system that translates thought into intelligible and recognizable speech through detections of brain activity that ultimately reconstruct the words a person hears with unprecedented clarity. The system can help people who cannot speak, those living with amyotrophic lateral sclerosis (ALS), or individuals recovering from stroke.

Concept illustration (stock image) via Science Daily  Credit: © adragan / Fotolia

"Our voices help connect us to our friends, family and the world around us, which is why losing the power of one's voice due to injury or disease is so devastating," says Nima Mesgarani, PhD, the paper's senior author and a principal investigator at Columbia University's Mortimer B. Zuckerman Mind Brain Behavior Institute. "With today's study, we have a potential way to restore that power. We've shown that, with the right technology, these people's thoughts could be decoded and understood by any listener."

Previous research has confirmed that when someone speaks or even thinking of saying something—patterns of brain activity begin to surface. Early efforts of the study focused on decoding brain signals through the utilization of computer models that analyzed spectrograms (the visual representations of sound frequencies). However, this approach has failed and so the study shifted to the use of a ‘vocoder’ which is a computer algorithm capable of synthesizing speech after being trained on voice recordings of human conversations

"This is the same technology used by Amazon Echo and Apple Siri to give verbal responses to our questions," said Dr. Mesgarani, who is also an associate professor of electrical engineering at Columbia's Fu Foundation School of Engineering and Applied Science. "Working with Dr. Ashesh Dinesh Mehta (co-author of the study), we asked epilepsy patients already undergoing brain surgery to listen to sentences spoken by different people, while we measured patterns of brain activity," said Dr. Mesgarani. "These neural patterns trained the vocoder."

Source: The Zuckerman Institute at Columbia University

About the Author
  • Nouran earned her BS and MS in Biology at IUPUI and currently shares her love of science by teaching. She enjoys writing on various topics as well including science & medicine, global health, and conservation biology. She hopes through her writing she can make science more engaging and communicable to the general public.
You May Also Like
AUG 26, 2020
Technology
Could a Selfie Be The Next Screening Tool for Heart Disease?
AUG 26, 2020
Could a Selfie Be The Next Screening Tool for Heart Disease?
We may soon live in a world where sending a selfie to your doctor can help with diagnosing heart disease—at least ...
SEP 04, 2020
Technology
Machine Learning May Protect Future Pregnancies
SEP 04, 2020
Machine Learning May Protect Future Pregnancies
It is critical for doctors to examine a placenta after the birth of a baby to determine a mothers health risks for any f ...
OCT 23, 2020
Space & Astronomy
Astronomers Merge AI and Photonics to Find New Earths
OCT 23, 2020
Astronomers Merge AI and Photonics to Find New Earths
Researchers from the University of Sydney have developed a new type of sensor capable of measuring and correcting starli ...
OCT 23, 2020
Chemistry & Physics
The Ever-Evolving Battle to Fight Corrosion in Nuclear Reactors
OCT 23, 2020
The Ever-Evolving Battle to Fight Corrosion in Nuclear Reactors
Since its birth in the early 20th century, atomic research has brought mostly positive impacts to our lives. This week i ...
DEC 08, 2020
Genetics & Genomics
Advancing Genetic Sequencing with Better Computational Tools
DEC 08, 2020
Advancing Genetic Sequencing with Better Computational Tools
The many advances that have propelled the field of genetics forward have taken a tremendous amount of work in different ...
DEC 22, 2020
Technology
New RNA Discovery Tool Accelerates Drug Discovery
DEC 22, 2020
New RNA Discovery Tool Accelerates Drug Discovery
Some diseases, such as metastatic breast cancer are known as “undruggable” or best described as unresponsive ...
Loading Comments...