Starry eyed: Where game graphics go next

[This feature was published in the September 2013 edition of Develop magazine, which is available through your browser and on iPad.]

The face of the next-gen ready centurion adorning the image above is made out of 10,000 polygons, while the full model notches up an impressive 150,000. On 360 and PS3, entire characters rarely push beyond 30,000, which gives you some sense of the graphical fidelity players will enjoy in the coming generation.

Nowhere has the pursuit of realism been more ferocious, more costly or more fragile than with games’ graphics. It started in the 90s with the switch to 3D, and since then cutting-edge visuals have become a top priority for games studios. In the last era especially, visuals in games have come close to matching that of animated films.

Now, as the next generation approaches, Develop is here to address the tough questions about graphics and rendering in games. We wanted to know how close games are to true photorealistic characters and environments. And why the uncanny valley still poses a threat to games. And whether the fixation with visual realism is here to stay at a time when players expect more interactivity than ever.

IMMERSIVE WORLDS

“What you want as a developer is to give players that ‘wow’ moment when they first enter a new world. That is achieved through better graphics, more dynamism and a compelling reason to be there,” Chris Doran, founder of lighting outfit Geomerics, tells us. “If we get that right, players will flock to the next generation of games.”

As Doran says, graphics are often the first thing that draws in the player. In spite of huge advancements, games are still viewed as the poor relation to Hollywood feature films. Those closest to the field are confident that photorealism remains one of the chief elements that will dispel the emotional barrier between games and audiences.

Dave Cullinane, account director at CGI production house Realtime UK (creators of the War Thunder images in this piece), believes games have a lot more to give beyond shooting and explosions: “The advent of true photorealism, which is now within touching distance, will bring a much greater depth and subtlety to games that can only benefit the medium. It will serve to grow gaming’s appeal and underscore the importance of the medium. Photorealism will help true character performance to flourish, helping elevate the emotion and engagement that many games already offer.”

SO CLOSE

So just how close to ‘true’ photorealism are we really? The father of Unreal Engine, one of gaming most capable graphics tools available, Tim Sweeney, said during this year’s Develop Conference that it could be as little as ten years or less for us to see photorealistic graphics that are “indistinguishable” from reality.

Jorge Jimenez has spent years researching real-time graphics and photorealistic rendering. And he agrees with Sweeney’s prediction.

“I believe that in the next few years we will see the barrier between games and film vanish. For example, by using baked lighting more extensively – in the cases where memory is less of a problem, we can take more power from offline renderings into our real-time realm.”

Jimenez, who is currently a technical director at Activision Blizzard and one of its core R&D graphics experts, feels that the problem of realising photorealism is actually about the application of skills, rather than simply having more horsepower.

“It is not to say that you do not need more power, which you always do. But rather, that the main obstacle to photorealism is in how and where to direct the resources you have available. And we have much to learn from the artists that are already creating photoreal offline renders; as shown in cgsociety.org galleries),” says Jimenez.

For Doran, Sweeney’s forecast is achievable, but he argues that ‘reality’ is not what players are after: “If you replace the word ‘reality’ with the word ‘film’, then I totally agree. It is a small point, but gamers don’t want games to look like reality; they want them to look like a live-action film.

“What cinematographers do is take a live-action set, and make it look hyper-realistic, for want of a better word. They make it look like a film. This is achieved through a combination of lighting effects and post-processing. These techniques are now in reach for next generation games developers. So I do expect some titles on PS4 and Xbox One to be indistinguishable from film.”

Some developers, however, are less sure that this will be the case.

Crytek’s US business development manager Sean Tracy, for instance, whose studio is behind the glorious-looking Roman warrior overleaf, as well as the technically bold Crysis series, admits that it is tough to pin down a timeframe for a goal that developers have long been chasing.

“I find it very difficult to predict more than a few years down the road to be frank,” Tracy tells us. “Surely, rendering and graphics will continue to evolve and improve, and I do think that at some point there will be a convergence where real-time rendering meets or exceeds reality. Whether that’s ‘ten year from now’, I simply don’t know. What I do know is that we don’t want to wait that long.”

Masaki Kawase, lead software engineer and shader architect at Silicon Studio, the Japanese firm whose post-processing middleware Yebis 2 is used in the likes of Final Fantasy XV and Valhalla Knights 3, has his doubts too.

“For pure rendering it might be true. The human eye might not distinguish the difference between still images like pictures compared side by side. However, as soon as humans interact with photorealistic graphics, they would look artificial rather quickly,” says Kawase.

“The Xbox One and PS4 are powerful systems capable of delivering stunning visuals, but I don’t expect that we will see graphics that are indistinguishable from reality until future console generations.”

LET’S GET PHYSICAL

Kawase’s comments will be an uncomfortable truth for some. However, just as the shader architect says, convincing gamers that they are in a believable world is not just about fooling them with stills that hold up to close scrutiny.

Characters and inhabitants must act and move in ways that feel natural. One of the core advancements that is setting the foundations for big leaps in graphics and animation in the coming generation is physically-based rendering.

Cited by many in this article as one of the key trends at this year’s SIGGRAPH, this technique allows programmers to model materials by using physical, measurable properties. Jimenez says that this new approach to algorithms will put the focus on measurements, which he feels is where the future of graphical advancements will start.

He explains that when authoring an asset, there are too many dimensions to search for: diffuse and specular albedo, specular gloss, subsurface scattering properties, and so on. Narrowing this search space by measurement will enable easier creation of photoreal assets.

This, in turn, should mean that programmers and artists are free to reach for greater levels of spectacle, such as more destructible environments. And greater emotiveness too should come with this, as the limitations on character interactions loosen.

Tracy tells Develop that a number of physically-based algorithms are behind graphical improvements to lighting and materials in Crytek’s game engine, which is powering Xbox One launch title Ryse. Using its physically-oriented lighting system, Crytek is attempting to ensure that every object in the game behaves as it should in the real world. And Crytek is not alone in viewing lighting as a core piece of the photorealism puzzle.

“In my opinion, the biggest challenge for the next generation of games is to improve the lighting to be consistent, by means of better global illumination. To make everything in the game feel integrated and sit well with their environments,” remarks Jimenez.

“Also, as more advanced antialiasing techniques are adopted, like CryEngine’s SMAA T2x, or the new Unreal Engine temporal antialiasing approach, we will see game renders to more closely match CG as it is seen in films.

“It was stunning to see the effect of good antialiasing in the real-time SIGGRAPH Unreal Engine Infiltrator presentation – turning it off completely ruined the CG film look and feel.”

Doran agrees that lighting will play an increasingly important role, though he also points out that developers must not “over-do it”, as has been the case in the past with effects such as bloom or normal mapping.

Instead, he says that: “it is much more about artistic judgement, and much less about coders trying out the latest idea they heard about at SIGGRAPH. The more control the art director has on the final image, the better the experience will be”.

ESCAPING THE UNCANNY VALLEY

New techniques such as physically-based algorithms mean that graphics may soon have the power to command the attention of even the most stubborn of non-gamers.

However, this still may not be enough for games to escape the uncanny valley. The term is used to describe viewers feeling a sense of unease or detachment to the appearance and movement of human-like characters, and cyberpsychologist Berni Good warns that games risk disengaging players if they fall into this trap.

Faceware is behind highly detailed facial capture animation technology used to generate such digital feats as Brad Pitt’s face in The Curious Case of Benjamin Button. Its VP of business development Peter Busch explains the response one of its CG shorts received: “We learned on the Emily Project in 2008 that there will always be sceptics, no matter how ‘real’ you get with your graphics. Our technology was used in combination with Paul Debevec’s Lightstage to create one of the most successful photoreal humans ever created.

“We cheated a bit because we only replaced the face of Emily – her clothes and hair were real. Even then, we received many comments that her hair looked ‘off’ and the cloth didn’t move realistically. It was reality and people still questioned it.”

The reaction to Faceware’s video short would be welcomed by those claiming that the pursuit of convincing, photorealistic visuals is a vain conceit. But, speaking pragmatically, the argument that games visuals are as good as they need to be is one that segments of the gaming audience and some developers themselves seem to have subscribed to.

“Good games need good gameplay, and photorealistic graphics won’t make a bad game into a good game. Indies are getting more attention from gamers, not because of graphics, but because of interesting gameplay,” acknowledges Silicon’s Kawase.

Though he accepts this outcome, he remains committed to the idea that more photorealistic and immersive effects will lead to more real emotions, which will be “an opportunity to change the game experience”. Kawase continues: “More powerful graphics means you can do more, but it means costs are going up too.

“So it is crucial to make realistic visuals less expensively and in a shorter amount of time using better tools and better algorithms, expending possibilities to work with more assets and more people on a team.

“We have more power but we need to use it differently than we used to. Compute shaders are still underused and we need to rethink all algorithms. The thing we need to focus on is how we can efficiently use compute shaders to make more realistic graphics.”

Still, Doran argues that pushing for better visuals should not negatively impact games in the next generation, provided that developers leverage technology and resources sensibly: “If you pick your battles carefully, and bring in technology from outside to help, there is no reason to compromise. I don’t think it would be smart for a developer to sit there today and think, ‘I’m focussing all my efforts on AI, so I won’t bother with graphics’.

“What they should think is: ‘AI is our defining feature for this title, so let’s go licence in the best graphics technology out there’. That is how it’s done in the film industry. With all the computing power available in next-generation consoles, and the added availability of cloud resources, there shouldn’t be a need to make compromises.”

NOT GIVING IN

During the reveals of PS4 and Xbox One, there was a noticeably lukewarm reception to the graphical leap for early next-gen titles. This was fuelled in part by the alluring visuals the likes of Naughty Dog, BioWare and Irrational have managed to squeeze out of the aging, eight-year-old current-gen consoles, just as their successors turned up to hijack the party.

Elsewhere, the appeal of quick-fire mobile games has pulled in millions. 3D graphics are only just beginning to be explored in more detail, but the staggering numbers for Angry Birds and Candy Crush Saga suggest that simple, 2D experiences will continue to thrive for many years yet.

Does this mean the fixation with high-end graphics is over? Hardly. But it does suggest that despite much progress, early next-gen titles may fall short of players’ expectations.

This is not the first time such as thing has taken place, as Busch says: “It will still take many years for the new hardware to be optimised to its fullest potential, much like when the PS3 and Xbox 360 came out. The expectations are higher than ever, but the trend will be an incremental increase in quality with each new title that each dev releases. My gut is that by the third release on new hardware, the games will really push the envelope when it comes to graphics.”

Regardless of whether graphics truly are on track to be indistinguishable from reality this decade, it is tough to deny that next generation video games are closing fast on the quality bar set by CG films.

Technology will be central, but the consensus among the practitioners Develop spoke to is that artistry and better methodologies will drive the graphical breakthroughs of tomorrow.

A final thought from Tracy on the matter: “We have a saying at Crytek that hard takes a day and impossible takes a week, usually said along with the tag phrase, ‘disrespect the impossible’. This is embedded in our company’s philosophy and thus it would be safe to say that we will never be satisfied with the status quo and will never stop pushing the boundaries of what’s possible in real-time rendering.”