AR and ChatGPT could be a killer combo for translation and transcription

OPINION: Wandering the halls of MWC 2023, I got the opportunity to go hands-on with Qualcomm’s reference AR headset design. While the hardware is very much still in development and focused mainly at manufacturers and developers, the demo introduced something I’d never considered before: leveraging the open AI platform ChatGPT. 

Qualcomm won’t actually be producing AR headsets for consumers, but the company has released a few reference AR headset designs over the past couple of years that essentially serve as a blueprint for other companies to create their own AR headsets. 

They’re slowly getting more compact and less like the bulky Microsoft HoloLens, but they essentially work in the same way, augmenting the real world with virtual elements. 

Manufacturers are already starting to leverage this tech – the Nreal Air is a good example of an early pair of AR glasses – but there’s still a long way to go before we’re in a fully augmented future with slimline AR glasses. The headset I used, for example, still requires a wired connection to a smartphone to process all the data from the headset. 

That has changed with Qualcomm’s latest AR Viewer Concept powered by the Snapdragon XR2 chipset that uses Wi-Fi 7 for super speedy connectivity – Qualcomm claims less than 10ms latency – but due to the sheer amount of wireless devices at the show, it wasn’t possible to demo it. 

Still, I was able to get some time with the slightly older Snapdragon XR1 reference headset, and I was presented with a new real-time translation and transcription service powered entirely by ChatGPT. 

ChatGPT is the hot topic in tech at the moment, with the open AI platform seemingly able to do anything from generating lines of code to speed up app development to helping kids do homework. There’s even a story about ChatGPT passing step 1 of the Medical Licensing exam in the US, showing just how impressive the hardware is.

With that in mind, ChatGPT seems to be the ideal candidate for real-time transcription and translation with a focus on natural language modelling. 

The headset used a dedicated microphone to pick up audio from the (very loud) demo area and near-perfectly transcribed what was said on the glasses, and it’s smart enough to detect multiple speakers and split the transcription between them, with speaker 0, speaker 1 and so on.

There is still a slight lag while the API does its thing, but within a second or two, I was seeing what was effectively live subtitles of my three-way conversation. 

That might sound a bit gimmicky at a first glance, but it could be a real game-changer not only for translation – imagine being able to talk to someone in a different language using only a pair of glasses – but arguably more importantly, a real boon for those that are hard of hearing. The ability to effectively transcribe in real-time could be freeing for those that are hard of hearing, especially if conversing with someone that isn’t fluent in sign language. 

Of course, there’s still a ways to go until we’re all wearing AR headsets with ChatGPT-enabled transcription and translation services, but it’s a tantalising preview of what we should really expect to see hitting the market in the next few years.