MAY 25, 2019 1:02 AM PDT

New language learning app uses smartphone camera and computer vision to teach vocabulary

WRITTEN BY: Nouran Amin

As part of a project from Microsoft’s in-house incubator Microsoft Garage, recent interns have created a new language learning app that utilizes the smartphone camera to enhance English literacy in adults by helping them learn the words for the things around them.

The app, a project from Microsoft’s in-house incubator Microsoft Garage, will initially be made available for testing and feedback for select organizations. Those who work with low literacy communities a

The app, Read My World, allows the user to take a picture to learn from a library of more than 1,500 words. A photo can be of real-world objects or a text found in a document and uses a combination of Microsoft Cognitive Services and Computer Vision APIs for identification purposes.

Although the app is not meat supplement formal classroom training, it offers a unique to learn some words for those who didn’t have the opportunity to participate in a language learning class.

“Originally, we were planning more of a lesson plan-style approach, but through our research and discovery, we realized a Swiss army knife might be more useful,” said Nicole Joyal, a software developer intern who worked on the project. “We wound up building a tool that can help you throughout your day-to-day rather than something that teaches,” she said.

Read My World will show the user the word’s spelling and phonetic pronunciation of the identified vocabulary. Photos can be saved to a personal dictionary in the app for later reference. Finally, the app has vocabulary games to encourage the practice of learned words.

Source: TechCrunch

About the Author
Doctorate (PhD)
Nouran is a scientist, educator, and life-long learner with a passion for making science more communicable. When not busy in the lab isolating blood macrophages, she enjoys writing on various STEM topics.
You May Also Like
Loading Comments...