Designing for HDR

From a consumer point of view, this can all be a bit confusing. What is HDR? What are the different standards? What does it mean when I look to buy a new TV? For gaming development though, it is something that really helps to make everything much more colourful and, potentially much more interesting, even more so than developing for 4K.

Both Sony and Microsoft now offer HDR gaming on their devices for compatible televisions but Microsoft’s Xbox One S was the first to really promote the technology for the console market, and had a line up of exclusive games designed to take advantage of it. Those games were Playground Games’ Forza Horizon 3 and The Coalition’s Gear of War 4.

Whilst we may think, especially from the outside, that HDR is a relatively new concept and at the upper end of television hardware, both studios have been utilising the technology for quite a while. “We’ve were already using an HDR render pipeline on previous games, says Jamie Wood, lead lighting artist at Playground Games. “Being able to output to HDR displays in Forza Horizon 3 was something new though. Our content was largely already suited to this, as we had taken great care in production to ensure there was a physically based hierarchy between all of our content. This includes light sources, emissive textures, cast dynamic light, the stars, the moon, sunlight and shadow. All of this content has to be balanced in order to look right when you start revealing brighter elements on HDR displays. A new calibration screen allowed us to determine the max Nits of the player’s display and fit the larger range of displayed luminance comfortably onto a given TV’s capabilities.”

This is also the case at The Coalition as studio technical director, Mike Rayner, explains. “The majority of modern AAA game engines have full HDR rendering pipelines and Gears of War 4 is no exception. From a development standpoint, we were already well positioned to add support for HDR.

“The biggest difference to our development was reviewing the game in HDR with the lighting, visual effects and UI team. This helped us to ensure our brightness ranges were tuned and content was calibrated to correctly leverage the full HDR range and achieve the creative vision we had for the game.”

We want to transport the player in to an immersive experience

Jamie Wood, Playground Games

STARTING UP

For studios looking to use HDR in future projects, there are invariably questions about how much extra work, and cost, it takes to produce HDR quality. “It doesn’t have to be a huge amount of work if you’re already using HDR rendering and PBR,” says Wood. “There will usually be a bit of content balancing and as mentioned before you need to work out how you are going to tonemap and output for HDR displays. The other part of standards like UHD to consider is the wider colour gamut.”

“Once we added the HDR output path, everything – for the most part – just worked,” adds Rayner. “We added HDR fairly late into the development process, however, since our engine is internally HDR and correctly calibrated, we did not have to re-touch a lot of art.”

One of the things with creating games with HDR support is that it isn’t just an add-on feature for a game. It normally needs addressing very early on in the process. “When you start to break down how to create compelling HDR images on a screen it’s a complicated task,” Wood explains. “Part technical, part artistic, part psychology. You need to be very careful about how you author and review work.

"The human eye adapts to changing light conditions, and how we perceive images can change dramatically depending on viewing conditions. This is particularly tricky with HDR as it’s quite a new technology and TV displays can vary quite dramatically – it’s harder to control the experience that our audience will have. We’re not even able to reproduce the full rec2020 colour space on any commercially available displays yet.”

Claude Marais, The Coalition’s Xbox graphics engineer, also points out the display variance issues. “The biggest problem we ran into was seeing banding in the darks for UI and loading screens,” Marais explains. “HDR TVs can not only show more details in the brights, but also in the darks. 8-bit content created on SDR displays might have banding artefacts in the very dark areas, but you simply cannot see it on a SDR TV/monitor, on HDR TV you can. We had to add a small contrast adjustment to hide these artefacts. This isn’t a problem with the scene rendering, that’s floating point or 10-bit.”

ILLUMINATING RESULTS

However there were benefits to using HDR in creating a game’s lighting, with very little extra work required. “We actually found that things like Bloom and God Rays translate well into HDR without any rework,” says Wood. “Global illumination falls under the category of something that should be well balanced for SDR output or HDR so there’s nothing additional we had to do there. For skies – like anything else you will perceive more of the bright, saturated values of the HDR sky image when viewed on an HDR display.”

Colin Penty is the CG supervisor at The Coalition. “Skyboxes were definitely a big focus point,” he explains. “You almost always want the sun to be the brightest thing in the world and that wasn’t always the case until we re-calibrated in HDR. Global illumination and crepuscular rays mostly just worked in the HDR space. Specific to the bottom end of the global illumination, we had to determine what that should look like in HDR and how to make it visible in HDR space. This was mostly a rendering-focused effort and less about content.”

As always with any kind of video game design, careful planning is important, as is research. “The other big content change that benefited our HDR support was our new HDR time- lapse sky technology,” says Wood. “We took the time to go to out to Australia (where Forza Horizon 3 is set) and capture full 24 hour HDR time-lapse skies, capturing the entire dynamic range of the sun and sky over the course of the whole summer.

This meant we could then project these skies in-game and bring a real sense of the true Australian lighting to our players, in full high dynamic range. We went to great lengths to capture our own sky content for the ambitious HDR time-lapse sky system in Forza Horizon 3, which meant we had full control over the quality and consistency.”

All of this of course helps the workflow although as Wood explains, it isn’t really anything out of the ordinary. “I would certainly recommend taking HDR bracketed reference photography. But I probably would have done regardless of HDR output. Getting those all-important hierarchies of lighting values correct is aided greatly by having multiple exposures in reference images. It’s hard to tell from a single exposure or a video just how much brighter a stadium floodlight is vs. a streetlight. The HDR reference allows you to get a sense of how your game camera exposure should impact the scene.”

Of course when you are making a game and you decide to create HDR visuals, the benefit has to go to one person, and one person alone – the player. “For us, we offer realistic visuals and HDR is just one of a number of tools to enhance that realism,” says Wood. “We want to transport the player into an immersive experience and, when done right, richer colours and larger contrast ratios closer to what you find outside in the real world will help to create those convincing realistic visuals.”

The Coalition’s Marais agrees. “HDR preserves details in the darkest and brightest areas of a picture that are lost using current standards so, for example, metals look more metallic. But more than that, it can also make the player have a more immersive experience. A good example is bright flashes from explosions – sudden bright flashes can cause a physical response where your eyes squint, you get startled, more adrenaline pumps through your body. Together with great audio effects, it can make you feel more like you’re actually in battle.”