Sky Sports and bringing virtual crowds to life in Atmos

What is football without crowds? With the Premier League thought to be one of the most exciting football leagues to watch anywhere, a part of that is the atmosphere fans bring to games.

And so when the pandemic hit, fans were required to stay away – but without their presence matches lose a sense of colour, impact and most importantly, noise.

Which is why Sky set about bringing some semblance of the football experience back with Project Restart’s virtual crowd noise. Ahead of the 2020/21 season, Sky’s Kevin McCue elaborated on how they did it.

Related: How Sky is bringing Dolby Atmos to the masses

“What can we do to enhance the experience?”

The road to the eventual solution was not always clear. While it seems rather obvious now, other ideas were in the pipeline.

Sky had been approached by companies about the concept of live crowds in an empty stadium. One idea was to use a stadium’s PA array to playback crowds during a live broadcast. Another decidedly more sketchy idea, was that fans could use an app where “people could shout at their phone and it would come out in the stadium”. For anyone who’s sat among fans, you’ll know exactly the type of ‘creativity’ they can come up with.

And (unfortunately) Sky was aware too. With no guarantee there wouldn’t be any profanity, McCue, head of Technical Innovation and Facilities at Sky, said “we felt very uncomfortable, this idea of essentially unleashing a group of people on our live broadcasts that we have absolutely no control over and anything could happen. And for various reasons, including things like transmission delays, and all this other kind of stuff, it really wasn’t going to work.”

So Sky reached out to EA Games in Vancouver, Canada. Sky already had a long-standing relationship, recording the sounds of crowds used in the FIFA football games. Both had similar ideas as to how they’d create a dynamic crowd environment, potentially through AI, computer vision (training computers to understand the visual world) and other crowd interactions.

However, latency issues made that all but impossible. So the solution, to dreadfully paraphrase Indeep’s 1982 classic, was that a DJ came to save virtual football crowds (life).

“It very quickly became a little bit of a tumbling snowball”

Despite doubts as to whether the concept of pre-recorded crowds would work, after a few demonstrations the head bods came round and then it snowballed, with the concept applied to the English Football League and set to be used for the upcoming NFL season.

Working with EA and another company in Vancouver that produces soundscapes and assets for gaming giant, the elements that would become the crowd mix were collated in DJ application called Ableton Live. Perusing the Ableton website, Live looks like a sprawling and thoroughly complex piece of software to a DJ layman like myself. So it’s a stroke of fortune that Adam Peri, Sound Supervisor at Sky Production Services, was in the mix, so to speak.

Peri had experience with Ableton and had done a bit of DJ-ing himself, and McCue describes Peri’s process as spending “the first week at home with a laptop that we found with Ableton on it and a MIDI keyboard. At the end of that period of time he’s like, ‘right, I need a mixer, I need some buttons. I need this, this, this, this and this, I think I can make this work.’ We supplied him with the kit and off he went. He did the first couple of events himself.”

As things began to align with the technology, method of playback and how it was going to be used in place, Sky also had “to train a whole bunch of people that maybe have no experience of Ableton, or no experience of playing a crowd live and no way really to rehearse it to, to train all these people and go live with it.”

As McCue concedes, “there was [sic] a few mistakes. I think we triggered a goal when it was a miss”, but that “over time it became very refined and actually became a really integral part of our broadcasting.”

“It is very hands-on, very dynamic. There’s no algorithms per se and there are lots of different loops, and what [the] sound supervisors are doing, blending those loops on top of each other, and then having these kinds of specific events that they can trigger on demand.”

“It went from something flat to something very immersive and wrapped around you”

But Sky is also broadcasting Premier League football in Dolby Atmos. They adopted the object-based audio tech several years back, and have pushed its use from art shows to live music events. In the context of adding virtual crowds, that added a little more complexity.

Usually, matchday sounds are captured with mics positioned around the pitch, with mics also in the gantry and pointed back at crowds to create a sense of height, as well as layering in discreet effects (such as PA announcements). Without physical crowds, there’s a space that needs filling.

And so with the crowd chants mixed into the Ableton Live workflow, the team essentially lifted the crowd noises and as McCue explains “we kind of panned to them so they existed in the 5.1 realm, but also existed in the height realm, and what that did was give you a bit of a lump of immersion above you… it gave you that enveloping kind of singing crowd [that’d] you’d expect at a football match because that’s where a lot of the a lot of the noise comes from.”

And by sneaking in some localised PA announcements into the heights, McCue calls the difference “night and day”, with the “echoey ropey old stadium PA above your head” making you feel as if you’re at the venue.

Related: Best Dolby Atmos soundbars

Considering the initial reaction to virtual crowds was not unlike a bad rash, a good chunk of football fans have eventually been converted. There have been some occasions – with Sky and other broadcasters too – where I’ve forgotten there aren’t any physical crowds there.

And we’ll be hearing more of Sky’s work, taking what worked and what didn’t with Project Restart, and applying that to its virtual crowds as the new season gets up and running. You could even see it applied to other sports, such as boxing, in the near future.

While it’s undoubtedly a compromise, it’s an innovation that’s making the somewhat sombre feeling of not being in stadiums just that little less keenly felt.