Augmented reality glasses display subtitles as people speak

Josh Feldman has relied on lip-reading his entire life to understand what someone is saying – but this week a new product that provides real-time transcriptions allowed him to follow a conversation without even closing the other person’s face see.

Born hard of hearing, Feldman has adapted to lip-reading while socializing. But it can be difficult to keep up in a world that often doesn’t accommodate the deaf or hard of hearing, especially during a pandemic where face masks have been a necessity.

Enter the first glasses with custom subtitles: Nreal Smart Glasses.

Feldman is one of the first to preview the glasses. He was sitting on a sofa wearing what appeared to be thick black sunglasses and was conversing with a man sitting to his right.

“I guess you mostly ask people what did someone say, you know?” the other man asked.

“One hundred percent,” Feldman replied without turning to look at him. “It’s about the changes that come with hearing loss.”

How was that possible? As the other man spoke, his words appeared in white type on the inside of Feldman’s glasses, allowing him to read and respond in real time.

“People who are hard of hearing would understand the feeling I had as I had this conversation, which lasted about five minutes,” Feldman said. “And I didn’t even look at the person on my right, I just looked ahead — it’s a life-changing moment.”

A UK startup called XRAI Glass is behind the technology that could allow millions of people who are deaf or hard of hearing to see in-person conversations they can’t hear.

The glasses use augmented reality (AR) technology, and a phone app provides the live transcription. Currently, the app is only supported on Android, but the company is trying to get approval for iPhone use as well.

“To have a conversation while not looking at the person you’re talking to,” Feldman said. “Wow. Wow.”

He can also read conversations he’s having on the phone when the phone is on speakerphone so the glasses can pick up the sound, and according to the company’s website, it can translate any language.

The concept is particularly useful during the COVID-19 pandemic: those who wear face masks cover their mouths, making lip reading impossible.

“Having something like these goggles, where the barrier of the mask is broken so they can actually see what’s being said… that’s really effective and would really make a difference for someone,” said Teri Devine, Associate Director of Inclusion at the Royal National Institute for Deaf People.

The technology isn’t perfect – it’s less reliable in a group environment when people are talking over each other.

“I think the real breakthrough will be when we get smart contact lenses,” said Dan Scarfe, founder and CEO of XRAI Glass. “This is going to be the thing that actually revolutionizes space. Their very first prototypes are coming out and will become mainstream in the next two to three years.”

The glasses will be publicly available from September.

Read Also :

US News