JAN 31, 2019 12:57 PM PST

Translating Brain Signals Directly into Speech

WRITTEN BY: Nouran Amin

In Scientific Reports, neuro-engineers developed a system that translates thought into intelligible and recognizable speech through detections of brain activity that ultimately reconstruct the words a person hears with unprecedented clarity. The system can help people who cannot speak, those living with amyotrophic lateral sclerosis (ALS), or individuals recovering from stroke.

Concept illustration (stock image) via Science Daily  Credit: © adragan / Fotolia

"Our voices help connect us to our friends, family and the world around us, which is why losing the power of one's voice due to injury or disease is so devastating," says Nima Mesgarani, PhD, the paper's senior author and a principal investigator at Columbia University's Mortimer B. Zuckerman Mind Brain Behavior Institute. "With today's study, we have a potential way to restore that power. We've shown that, with the right technology, these people's thoughts could be decoded and understood by any listener."

Previous research has confirmed that when someone speaks or even thinking of saying something—patterns of brain activity begin to surface. Early efforts of the study focused on decoding brain signals through the utilization of computer models that analyzed spectrograms (the visual representations of sound frequencies). However, this approach has failed and so the study shifted to the use of a ‘vocoder’ which is a computer algorithm capable of synthesizing speech after being trained on voice recordings of human conversations

"This is the same technology used by Amazon Echo and Apple Siri to give verbal responses to our questions," said Dr. Mesgarani, who is also an associate professor of electrical engineering at Columbia's Fu Foundation School of Engineering and Applied Science. "Working with Dr. Ashesh Dinesh Mehta (co-author of the study), we asked epilepsy patients already undergoing brain surgery to listen to sentences spoken by different people, while we measured patterns of brain activity," said Dr. Mesgarani. "These neural patterns trained the vocoder."

Source: The Zuckerman Institute at Columbia University

About the Author
  • Nouran is a scientist, educator, and life-long learner with a passion for making science more communicable. When not busy in the lab isolating blood macrophages, she enjoys writing on various STEM topics.
You May Also Like
AUG 03, 2021
Technology
Using Snakes to Monitor Nuclear Radiation
AUG 03, 2021
Using Snakes to Monitor Nuclear Radiation
Nuclear disasters can be devastating. When these accidents happen, such as Chernobyl in the Ukraine or Fukushima in Japa ...
AUG 16, 2021
Infographics
How Can We Make Sure Artificial Intelligence Won't Destroy Humanity?
AUG 16, 2021
How Can We Make Sure Artificial Intelligence Won't Destroy Humanity?
Artifical intelligence (AI) is becoming increasingly pervasive in daily life. While still far from the robitic humanoids ...
AUG 06, 2021
Space & Astronomy
New Details About the Far Side of the Moon are Revealed
AUG 06, 2021
New Details About the Far Side of the Moon are Revealed
It takes 27,322 days for it to orbit Earth and 27 days for the Moon to rotate once on its axis. We can't detect that mot ...
SEP 01, 2021
Chemistry & Physics
New Study Suggests One-third of Binary Stars Have Engulfed Their Planets
SEP 01, 2021
New Study Suggests One-third of Binary Stars Have Engulfed Their Planets
While our solar system is relatively calm and inert, some sun-like stars out there will literally eat the planets in the ...
SEP 14, 2021
Clinical & Molecular DX
HIV Self-Test App Proves Promising
SEP 14, 2021
HIV Self-Test App Proves Promising
A new app allows users to self-test for HIV, which has proven to help positive patients get access to medical care and c ...
SEP 22, 2021
Chemistry & Physics
XENON1T Physicists May Have Directly Detected Dark Energy
SEP 22, 2021
XENON1T Physicists May Have Directly Detected Dark Energy
In what could be a revolutionary discovery, a team of physicists from the XENON Collaboration may have detected dark ene ...
Loading Comments...