Trusted Reviews is supported by its audience. If you purchase through links on our site, we may earn a commission. Learn more.

Alexa now has a ‘whisper mode’ and it’s way more complex than you’d think

During last week’s colossal drop of new products and features, Amazon announced a new whisper mode for its Alexa personal assistant. The feature is centred around the idea there are times of the day when you’re naturally quiet, such as early in the morning when your partner may still be sleeping, or during the evening when the children are asleep.

However, when others are catching some z’s you may still want to ask Alexa for an update on your commute, or to set an alarm for the morning, without waking fellow members of the household. So now, if you whisper your command at Alexa, she will whisper back in kind.

A week on, Amazon is explaining how the feature works, and it’s a lot more complex than it sounds. Amazon scientist Zeynab Raeesy says whispered speech is typically low energy and unvoiced, meaning it lacks the vibration in the vocal chords. That makes it much more difficult for a listening device like an Amazon Echo to pick up the sounds effectively.

Related: Which Amazon Echo should you buy

Raeesy says Amazon compared the performance of two different neural networks in order to to distinguish between words spoken normally and those whispered. They found a long short-term memory (LSTM) network performed better than multilayer perceptron (MLP) network

She wrote: “The models are trained on two categories of features. One is log filter-bank energies, a fairly direct representation of the speech signal that records the signal energies in different frequency ranges. The other is a set of features specifically engineered to exploit the signal differences between whispered and normal speech.

“We found that an LSTM network that doesn’t use handcrafted features performs as well as an MLP that does, indicating that LSTMs are capable of learning which signal attributes are most useful for whisper detection.”

Raeesy added there were caveats to this approach, namely that the more data the LSTM network was exposed to, the less improvement the handcrafted features offered. So, the model that now sits within Alexa doesn’t include the handcrafted features at all.

Other problems the team had to overcome included the “end-pointing” process. Usually, Alexa is able to detect the end of a command due to the period of silence at the end. This becomes more problematic with whispering and the ability of the LSTM network was less effective towards the end of utterances.

She said: “Unexpectedly, averaging the entire signal — including the troublesome final 50 frames — yielded the best results. We suspect, however, that that’s because the samples of whispered speech that we used in our experiments were manually segmented, while the samples of normal speech were automatically segmented, using Alexa’s production end-pointer.

“There could be some consistent difference between manual and automatic segmentation that the system was actually exploiting to distinguish the two types of input, and dropping the final 50 frames made that difference more difficult to detect.”

Do you think whisper mode is a useful addition to Alexa’s skillset? Drop us a line @TrustedReviews on Twitter.

Why trust our journalism?

Founded in 2003, Trusted Reviews exists to give our readers thorough, unbiased and independent advice on what to buy.

Today, we have millions of users a month from around the world, and assess more than 1,000 products a year.

author icon

Editorial independence

Editorial independence means being able to give an unbiased verdict about a product or company, with the avoidance of conflicts of interest. To ensure this is possible, every member of the editorial staff follows a clear code of conduct.

author icon

Professional conduct

We also expect our journalists to follow clear ethical standards in their work. Our staff members must strive for honesty and accuracy in everything they do. We follow the IPSO Editors’ code of practice to underpin these standards.

Trusted Reviews Logo

Sign up to our newsletter

Get the best of Trusted Reviews delivered right to your inbox.

This is a test error message with some extra words