MAY 25, 2019 1:02 AM PDT

New language learning app uses smartphone camera and computer vision to teach vocabulary

WRITTEN BY: Nouran Amin

As part of a project from Microsoft’s in-house incubator Microsoft Garage, recent interns have created a new language learning app that utilizes the smartphone camera to enhance English literacy in adults by helping them learn the words for the things around them.

The app, a project from Microsoft’s in-house incubator Microsoft Garage, will initially be made available for testing and feedback for select organizations. Those who work with low literacy communities a

The app, Read My World, allows the user to take a picture to learn from a library of more than 1,500 words. A photo can be of real-world objects or a text found in a document and uses a combination of Microsoft Cognitive Services and Computer Vision APIs for identification purposes.

Although the app is not meat supplement formal classroom training, it offers a unique to learn some words for those who didn’t have the opportunity to participate in a language learning class.

“Originally, we were planning more of a lesson plan-style approach, but through our research and discovery, we realized a Swiss army knife might be more useful,” said Nicole Joyal, a software developer intern who worked on the project. “We wound up building a tool that can help you throughout your day-to-day rather than something that teaches,” she said.

Read My World will show the user the word’s spelling and phonetic pronunciation of the identified vocabulary. Photos can be saved to a personal dictionary in the app for later reference. Finally, the app has vocabulary games to encourage the practice of learned words.

Source: TechCrunch

About the Author
  • Nouran is a scientist, educator, and life-long learner with a passion for making science more communicable. When not busy in the lab isolating blood macrophages, she enjoys writing on various STEM topics.
You May Also Like
APR 09, 2021
Cell & Molecular Biology
CRISPR-SNP Chip Finds Point Mutations in DNA Without PCR
APR 09, 2021
CRISPR-SNP Chip Finds Point Mutations in DNA Without PCR
Some diseases, like sickle-cell anemia and familial amyotrophic lateral sclerosis are due to errors in only a single let ...
APR 09, 2021
Technology
Ankle Exoskeleton Enables Faster Walking
APR 09, 2021
Ankle Exoskeleton Enables Faster Walking
As people age, their speed in walking often deteriorates which can be a frustrating experience. Engineers at Stanford Un ...
MAY 06, 2021
Clinical & Molecular DX
Using Biology as Technology: A Paradigm Shift
MAY 06, 2021
Using Biology as Technology: A Paradigm Shift
Digital networks have changed the world by integrating into almost every aspect of our lives, but they have yet to direc ...
MAY 06, 2021
Drug Discovery & Development
Tweaked Version of Ketamine Could Solve the Opioid Crisis
MAY 06, 2021
Tweaked Version of Ketamine Could Solve the Opioid Crisis
Medications that manage post-operative pain are some of the main entry points to opioid addiction and overdose in the US ...
JUL 07, 2021
Technology
New Device Inactivates Bacteria That Cause Ear Infections
JUL 07, 2021
New Device Inactivates Bacteria That Cause Ear Infections
Researchers may have developed a device that could offer new therapeutic benefits to ear infection patients. According t ...
JUL 11, 2021
Earth & The Environment
A Lake in Antarctica Suddenly Drains
JUL 11, 2021
A Lake in Antarctica Suddenly Drains
During the winter of 2019, a large and deep lake on the Amery Ice Shelf in East Antarctica was lost in about a week. Abo ...
Loading Comments...