Microsoft is using its HoloLens headset and AI technology to build a seeing headband for blind people, and it sounds similar to the next generation hardware seen on Star Trek’s favourite Lieutenant Commander, Geordi La Forge.
Editor’s note: Yes, the use of “next generation” was totally intentional. We regret nothing.
Microsoft detailed the research in a blog post today, giving examples of working trials it’s already running as a part of the firm’s wider ‘Project Tokyo’. The research uses a modified HoloLens and specialist software to let blind users know not just when someone is near, but who they are facing.
“The device has an array of grayscale cameras that provide a near 180-degree view of the environment and a high-resolution color camera for high-accuracy facial recognition. In addition, the speakers above the user’s ears allow for spatialized audio – the creation of sounds that seem to be coming from specific locations around the user,” explained the blog post.
“Machine learning experts on the Project Tokyo team then developed computer vision algorithms that provide varying levels of information about who is where in the user’s environment.”
Related: How to watch Star Trek Picard episode 1
The audio signals are reportedly still in development, but already let users know a variety of spatial information, as well as who is in the room with them.
“If the device detects a person one meter away on the user’s left side, the system will play a click that sounds like it is coming from one meter away on the left. If the system recognizes the person’s face, it will play a bump sound,” explained the post.
“When the user only hears a click but wants to know who the person is, a second layer of sound that resembles an elastic band stretching guides the user’s gaze toward the person’s face.”
The tech is still in its early days, but is reportedly part of a wider push by Microsoft to create a platform using AR and AI that can be customised to help a variety of different disabilities.
The tech document in the post is pretty chunky but Microsoft claims it is already creating further HoloLens prototypes based off early trials.
“The broader Project Tokyo research effort continues, including new directions in machine learning that allow users to adapt the system to their personal preferences,” read the post.
This has reportedly already led to design changes to make the headset less noticeable, indicating it could one day match the svelte design of the Star Trek tech.