High Frame Rate Video: What is HFR and why should I care?

Many experts predict that High Frame Rate video, or HFR, will be the next big thing in AV technology. But what is it and what does it mean for you? From watching live TV at home to going out to see a film at the cinema, our razor-sharp guide explains everything you need to know about HFR.

Watching TV shows and films is as much about what you’re not seeing as what you are. Video is delivered in single frames rather than a continuous image stream, meaning you’re always going to lose some image information in the ‘gaps’ between the frames.

The exact amount of real video information your eyes and brain get to work with depends on the number of frames a second the video uses. The fewer the frames, the more your brain will have to fill in the blanks – and the more likely you’ll feel motion-related issues such as blurring, judder and softness.

As other video technologies such as 4K and 3D continue to evolve, frame-rate-related problems have come under the spotlight, with a growing number of content creators from all corners of the film and TV worlds starting to address the issues.

Related: What is HDR?

HFR at the movies

The most high-profile moves into developing higher frame rates have come in the world of cinema. Peter Jackson got the ball rolling in earnest by releasing the first of his three Hobbit films in a 48 frames a second format – twice the normal 24fps rate films have stuck to for decades.

Director Ang Lee then ramped things up to another level in 2016, by releasing Billy Lynn’s Long Halftime Walk at a massive 120fps – the same frame rate James Cameron is reportedly looking to use for his Avatar sequels.

With such big-name players showing an interest in pushing the high frame rate cause, it must be a technology that really delivers the goods, right? Well, that depends on who you talk to. At least where films are concerned.

Reviewers of all the high-profile high frame rate film releases we’ve seen to date have been divided over whether upping the frame rate helps or hinders the viewing experience. Pretty much all agree that HFR improves the clarity and detailing in the picture, especially when viewing in 3D. But many reviewers find this extra clarity is more detrimental to the viewing experience than beneficial.

The problem for most HFR movie haters is that by making the image look crisper, cleaner and more fluid, HFR actually tends to make the action look more artificial – the exact opposite of what it’s intended to do.

The negative audience reaction to the first Hobbit film’s 48fps release was so widespread that director Peter Jackson added some gentle blurring technology to the high frame rate versions of the two sequels to tame the clarity of the original filmed material.

The response was even worse for the Ang Lee’s 120fps Billy Lynn. Pretty much every review of the HFR version of Lee’s film spoke negatively about the effect of the high frame rate. Slate columnist Daniel Egber said, “it looked queer, uncinematic, like a theater sketch acted out in virtual reality.” More pithily, film critic Bilge Ebiri tweeted after watching Billy Lynn’s Halftime Walk that “High Frame Rate is a… crime against cinema.”

Billy Lynn’s Long Halftime Walk is now available to buy on 4K Blu-ray in 60fps – the only film released in this format to date.  And even at this reduced (from its original 120fps) frame rate, it’s hard to deny that while those extra frames per second help to deliver possibly the cleanest, sharpest picture quality yet, they do also leave some of the acting and staging looking painfully artificial.

Related: Ultimate 4K Ultra HD Blu-ray guide

Director Ang Lee and Joe Alwyn on the set of TriStar Pictures’ Billy Lynn’s Long Halftime Walk

The bottom line for HFR movies seems to be that capturing real image data at 48fps (in The Hobbit’s case) or 120fps (in Billy Lynn’s case), and then playing back those frames at the rate at which they were filmed, leaves no hiding place for artifice issues. This is what seems to make HFR such a challenge in the world of cinema.

For what it’s worth, my own take on HFR in the movies isn’t as negative as that of most reviewers. I saw all three of the Hobbit films in HFR at the cinema, and with each film I found the experience more engaging. Perhaps this was in part because of Jackson’s ongoing efforts to improve the experience, but I think HFR may also be something you need to acclimatise to. In other words, the more experiences you have of it, the less jarring and more natural it appears to be.

There’s likely also an issue with film-makers needing to learn how to use HFR effectively. The sort of filming and special-effects techniques that work in 24fps just don’t work for 48fps and higher. (Although the Billy Lynn experience suggests that 120fps may be a stretch, no matter what a director does – unless James Cameron can prove the world wrong again.)

In addition, Billy Lynn’s extreme HFR efforts additionally revealed a bunch of pretty extreme technical headaches associated with HFR film-making.

For starters, as Ang Lee discusses in the extra features you get with on the Billy Lynn 4K Blu-ray, it turned out that you really can’t use any makeup on actors’ faces in HFR, because it’s too easy to see. Also, the actors had to be trained to act incredibly subtly, as any artificiality in a performance hits you in the face with HFR. Unfortunately, Billy Lynn reveals all too clearly that some of Lee’s actors accomplished this ultra-naturalistic acting technique better than others!

Lee also reveals that filming at the sort of HFR levels he used requires a huge amount of light to prevent the image looking too dark. As well as leaving many sequences in Billy Lynn looking artificially lit, this need for light hamstrung the extent to which Lee was able to move his camera around. This meant that a depressing number of sequences had to be shot using a largely static, locked-off camera – despite the stultifying effect this can have on the action.

Finally, Lee’s production team revealed that working with high frame rates means handling huge amounts of data. While making Billy Lynn in 3D, 4K HFR, Lee and his team had to store an estimated 40 times more data than has to be stored for a normal 2D, 2K, 24fps film. Anything that increases the amount of data film-makers have to work will create substantial headaches, delays and potential extra costs as the material moves between different production departments.

As with all things, however: where there’s a will, there’s a way. If HFR starts to win over more viewers, then rest assured that solutions to the problems will be found. The big question is whether movie-goers will demonstrate enough enthusiasm for HFR to sustain the expense and effort associated with delivering it.

Related: Best 4K TVs

HFR on your television

Given how effective HFR is at capturing the minutiae of reality, it makes sense that there’s lots of interest in using the technology for TV documentaries and, especially, sports coverage. After all, you don’t have to worry about HFR making things look artificial when what you’re filming real life – HFR is just capturing reality at more frames a second.

Using HFR for sport is especially tantalising, since anything that consistently features a lot of movement and camera pans will likely suffer more greatly from the information loss between video frames associated with lower native frame rates.

Not surprisingly, many of the TV world’s biggest players are jumping on board.

The BBC’s R&D department has been experimenting with HFR sports since at least 2014, and has publicly demonstrated Commonwealth Games footage playing at 100 true frames a second. In 2016 at the IFA technology show in Berlin, LG demonstrated tennis and athletics footage on a pair of specially created OLED TVs at 100/120 frames a second.

Indeed, in May 2017, LG partnered up with European satellite broadcaster SES to run demos of 4K HDR video being broadcast at 120 frames a second, proving that technically it could already be possible for broadcasters to start showing the likes of football, tennis and cricket in HFR for greater clarity and life-like fluidity.

Looking ahead, Japan’s NHK broadcaster has announced plans to show the 2020 Olympic Games in Tokyo in what it calls ‘full-featured 8K’ – shorthand for 8K resolution backed up by high frame rates.

While viewer reaction to HFR in movies is mixed, to say the least, reactions to HFR broadcasting – especially of sport – seem far more positive. In the case of the BBC’s Commonwealth Games test, for instance, only 5% of the public audience couldn’t see or weren’t impressed by the differences HFR made to the viewing experience.

At this point, you might well be thinking that your TV already shows pictures at 50 or 60Hz. Or if you have a fairly premium TV, it may claim to be doing motion at 100Hz, 120Hz, 200Hz, 240Hz or even much higher numbers.

So that means you’re already enjoying high frame rate pictures, right? Not quite.

Related: Black Friday

The thing is, TV refresh rates aren’t strictly the same thing as frames per second. Actually, the native video frame rate for TV material is 25 or 30 frames a second, depending on whether the footage is being made in Europe or the US.

The 50/60Hz figures you may be familiar with from the TV world refer to the way most TV broadcasting uses a process called ‘interlacing’. This is where only the even or odd lines of a frame are produced, but at a rate of 50 or 60 frames per second. In other words, you’re not seeing 50 or 60 frames of real image data every second, but rather 25/30 real image frames a second ‘manipulated’ into a 50/60Hz effect.

Televisions that claim to be 100Hz, 200Hz and so on are actually just using processing – usually known as motion interpolation – to calculate how extra frames of pictures should look. They then insert these ‘made up’ new frames between the 25/30 frames per second of original, actual image content.

Such motion interpolation processing can make images look free of judder and blur, as HFR is designed to do. But those processed extra frames won’t deliver the same degree of crispness and accurate detail that you’d get if there were 100 or 120 frames of actual image data in play.

All of which brings us back to the point that true high frame rate TV material – at least with ‘real’ content such as sports and documentaries – yields picture quality benefits that the vast majority of the viewing public who see it in action can readily appreciate.

It’s a pity, then, that the technical issues associated with HFR broadcasting are even more challenging than those associated with movies. Especially if you’re trying to deliver a live broadcast in HFR.

Looking at the BBC demonstration as an example, based on current camera technologies, the Beeb’s engineers were faced with capturing around 9GB of 4K HFR data every three seconds. They had to link eight high-speed SSD drives to provide 4TB of storage and deliver data over a quad-link, 3GB per second SDI interface to a PC. And this is all before you’ve even started to think about the need to broadcast and/or stream the sort of data quantities associated with 4K HFR images.

There’s one further big hurdle to the dream of high frame rate TV: the current state of TV hardware. At the time of writing, I don’t believe there are any TVs capable of handling true frame rates higher than 60fps.

The ever-troublesome HDMI connection is also an HFR problem, since current (v2.0) HDMI standards support bandwidths of 18Gbps, which is only enough to support 4K images at up to 60fps.

The next generation of HDMI connection, v2.1, will support bandwidths up to 48Gbps, will be able to carry resolutions of up to 10K and, more pertinently for this article, frame rates of up to 120fps. But HDMI 2.1 isn’t expected to start appearing on AV equipment until next year, plus you’ll also need to buy a new generation of HDMI cable to carry so much data.

Moreover, no TV manufacturer has – to the best of my knowledge – confirmed that it will be enabling its 2018 TVs to support the sort of maximum frame rates HDMI 2.1 makes possible. Even if any such high frame rate sources are available to watch by 2018, which there probably won’t be…

Is HFR the future or a false dawn?

Billy Lynn (Joe Alwyn), dancers, and Alabama State Marching Hornets in TriStar Pictures’ Billy Lynn’s Long Halftime Walk

There’s little doubt that HFR technology is going to play a big role in the future of film and television. It has the support of a number of Hollywood’s biggest names for movies, and its potential for TV – at least with ‘real life’ content such as sports and documentaries – is obvious.

However, what’s less clear right now is how soon we can expect to be able to gorge ourselves on mountains of high frame rate content in our living room.

Yes, the Billy Lynn 4K Blu-ray has already given us a genuine glimpse of what a high frame rate future might look like, despite actually only offering half the frame rate deployed for its theatrical HFR release. While often delivering objectively stunning picture quality, though, this release highlights at least as many HFR problems as strengths.

Also, I don’t know of any other film that’s coming to 4K Blu-ray in the same high frame rate, and James Cameron’s distant Avatar sequels are currently the only other films in the cinematic pipeline that look set to use HFR technology.

The reality-loving TV world has arguably the more compelling reason to embrace HFR in a hurry, but those ambitions are currently being thwarted by a mixture of hardware limitations at every stage of TV production and distribution.

In the end, it may actually fall to the gaming world to blaze the high frame rate trail. The benefits of higher frame rates have long been understood in the gaming world, with much of the push for ever more power from consoles and PCs being focused on improving frame rate consistency.

Related: HDR gaming explained

With PCs already pushing beyond 60fps for some game titles, it’s in this arena that we might see the most initial pressure to speed up the development of all the various hardware infrastructure that’s needed to take HFR from dream to ultra-crisp reality.