Interview: Umbra 3 and its next-gen visual trickery

Umbra is the game development equivalent of a magic trick. A trick intended to maintain the illusion created by those on-screen polygons and your point of view.

It is this tool that has helped to generate the striking vistas of Killzone: Shadow Fall’s sun-drenched cityscape. Concealed those pesky enemy combatants as they scurry between buildings in the middle distance on Call of Duty: Ghosts. And will soon be sustaining the chaos of flaying bodies, debris and dust particles in the destructible environments of Bungie’s sci-fi MMO, Destiny.

Finnish middleware firm Umbra Software is aiding all of these next-gen games and more with the third version of its self-titled visibility tool.

Develop spoke to director of developer relations Thomas Puha and programmer Sampo Lappalainen about what the visibility tools does for next-gen titles on PS4 and Xbox One, whether this specialist area of games middleware is underserved and what’s next for occlusion culling.

For our readers who are unfamiliar with your tool, what is Umbra, and can you describe the ongoing problem it aims to solve?
Thomas Puha: The Umbra 3 middleware is a visibility solution, it’s a tool that optimises rendering and level creation. Programmers need to integrate it into the engine, but it then affects the level creators workflow and improves production time. As each engine is different, Umbra cannot be a simple plugin, but you have to spend a bit of time integrating it into your engine and pipeline if you are using your own engine.

The technical term for the problem we solve is occlusion culling. What that means is making sure that objects in a 3D world that are hidden behind other objects do not get rendered.

That sounds easy, but ask any programmer, and they tell you it’s a very tricky problem to solve. There are a few of different solutions, but they often require a lot of manual work from artists or complicated CPU-GPU collaboration.

Umbra 3 basically automates the visibility solving process. The end result should be that you get great rendering performance, let artists concentrate more on creating the worlds and hopefully being able to put a whole lot more detail into them due to our tech.

With the arrival of PS4 and Xbox One, what challenges do developers face when it comes to occlusion culling for next generation games?
Sampo Lappalainen: With the new hardware there is a pressure to make the gameworlds larger and more complex in a very little time for launch especially. As the step up in fidelity is sudden and an order of magnitude in size, the existing technology will suddenly be put under a lot of stress. Many occlusion culling systems will simply not be able to handle this change and need a complete rewrite. Some systems rely on artist work so the larger worlds mean a lot more artists hours put into modelling occluder objects or portals.

When you are working on a launch title and there is all this new work you suddenly have to do, the temptation is strong to go shopping for specialised middleware that is already optimised for the new hardware. This is a huge opportunity for us as people are now actively on the prowl for new tech.

Umbra has already been selected to assist with several next generation titles, including launch titles Call of Duty: Ghosts and Killzone: Shadow Fall. What specific aspects of Umbra made these developers choose your tool?
TP: There are several factors. Obviously, first and foremost it has to be great, well-optimised technology that brings tangible benefits and provides something that’s so good that there’s no point in developing it yourself. Like Sampo says, when new machines come to play, you want to upgrade and change your tools, so it’s a good time to take a look at different solutions for doing things.

I’d like to think that Umbra has great reputation in the industry and Finnish programmers and developers in general have a good rep across the globe.

I wish I could tell you some of the conversations we’ve had with the Neversoft and Infinity Ward development teams about this, as they’ve been overwhelmingly positive about Umbra 3 and given us very valuable feedback.

Both cases are very different. We started working with the Ghosts development teams in early 2013, so it was a fast turnaround. But with Guerrilla, we started back in late 2011. S they’ve had to deal with our in-development software on top of the everchanging PlayStation 4 development environment, which it hasn’t always been easy. Whereas the Call of Duty team got to grips with a very mature version of Umbra 3.

Then you have a team like Remedy, back here in Finland, who we’ve worked with for years. They just trust our solution from one version to the next and go for it, as they know they can rely on us.

Also our work with Bungie for Destiny these last three years has definitely been a massive help as game developers have learnt about our collaboration. The guys at Bungie, like their graphics lead Hao Chen, have said a lot of nice things about our tech and what it’s like to work with us compared to other tool makers. So hats off to them. That’s been of critical help to us. Good personal relationships with the people you work with are always important.

Can you describe any scenarios in any of these titles that represent how Umbra 3 can aid next-gen games?
TP: The teams at Infinity Ward and Neversoft are super good at what they do and pushed us really hard. A good example is the Stonehaven map in Call of Duty: Ghosts, which didn’t work well using the traditional visibility solving methods they had, but required Umbra 3 tech to really work on next-gen. We helped by boosting the framerate there. We did get feedback that our software was especially useful in the later optimisation stages of development and gave big performance gains. For example they could go way crazier with shaders on next-gen.

In the Witcher 3, the folks at CD Projekt Red have a great team, but the size of the game’s world and the amount of content they have to generate does mean they need some help. We’ve custom developed some streaming features with them and it’s been a fun process for our engineers.

Destiny and Bungie are special because they were the first ones to sign up for what became Umbra 3 and they have had a lot of input into our development, so their engine is built around the strengths of our software and thus it’s used in many ways.

It’s important to understand that when it comes to visibility, a lot of times in the past, it’s work done manually by artists who set portals into the gameworld to handle visibility. Now, when you change such things at the end of production to optimise your game and content, it’s very risky because it’s very easy to break things at that point, which is something you really don’t want to do.

Our technology does visibility automatically so artists just play around with parameters, compute and see the results.

The tool is also capable of handling destructibles and user-generated content, which are said to become an even bigger part of games this generation. Tell us some more about this. Can you describe how an occlusion culling tool can solve the memory and performance demands such processes will mean for next-gen games?
SL: Umbra handles destructible content in a couple of ways. We provide a simple way to toggle objects ability to obstruct others on/off, so this provides an easy way to achieve destructibility. We have also implemented a CPU-based triangle rasteriser, which we boldly claim to be the ‘fastest of its kind’. We are quite confident that this is the case, specifically on PS4 and Xbox One. Triangle rasterisation allows you to do more complex dynamic stuff in the game.

User-created content is a bit more tricky as, even though Umbra 3 runs on any device, the actual offline data generation tools run on PC only. As the PS4 and Xbox One hardware is not much different from your regular PC, adding support for running the tools on the consoles is not too difficult. We are working on releasing this capability early next year.

Let’s go back a bit now. Middleware is often about maintaining the illusion for the player, but your field in particular is about a complex process of making things disappear and reappear. What led you to become involved in occlusion culling for games? And why?
TP: Well it’s a difficult problem to solve and we like solving difficult problems and coming up with different ways of doing it.

The company itself spun off from Finnish Hybrid Graphics when Nvidia bought it. Our CEO [Otso Mäkinen] wanted to keep developing the occlusion culling tech, so he set up Umbra to continue doing that, and that was back in 2007. The way especially Umbra 3 works, it generates very accurate spatial data so it has a lot of differing uses actually, such as audio occlusion, triggering scripted events and such. It’s actually got quite a few different uses outside of the core occlusion culling which we hope to bring out more in 2014.

Would you say occlusion culling is an underserved area of games middleware? If so, why do you think that is?
It is, which is good for us, obviously. Occlusion culling is very specialised and not exactly the sexiest thing to develop, I guess. It’s really very difficult to come up with an optimised solution. Not a lot of people know how to do it properly and tend to spend their time on other coding matters, of what there are plenty.

Also, by its nature, occlusion culling tech cannot be just a simple plugin. You have to do some coding to integrate it and take some things into consideration when doing it.

You’ve been developing occlusion culling middleware for at least eight years now. How has the field developed? And what has influenced your thinking for Umbra 3?
The hardware and games have changed quite a lot. Back in the day, Umbra was used in a lot of MMOs and still is. But now we are licensed much more for action games. That’s a sign of the times as well, of course.

While there’s less MMOs and the triple-A space is getting smaller, the explosion of mobile and Unity as a platform have been significant since we’ve started.

Our previous product Umbra 2 (aka Occlusion Booster) was using the GPU’s occlusion query functionality, which has inherent issues with syncing data between the CPU and GPU. That caused problems, so while OB was used in games like Alan Wake, Mass Effect 2 and 3 and a few others, it was not as successful as we hoped. We were late in the game, so to speak, on the PS3/Xbox 360 generation.

So when started designing Umbra 3, we developed it so that it would only require the CPU and that the visibility data would be generated offline instead at runtime. Being CPU-based makes it easy to port and keeps the GPU free. It was also a major strategic decision to be on next-gen consoles right from the start in as optimised fashion as possible. Working with teams like CD Projekt Red and Guerrilla definitely helped and kept us on our toes.

Are you catering to independent developer at all with Umbra 3? If not, why?
SL: So far Unity has been our channel for reaching out to indies. Not everyone wants to use Unity though and they want to work on their own tech instead. We’ve got some inquiries from such teams and companies recently and have been looking into some business models that would make sense for both us and the indie dev.

Our product is very technical. The devs typically have some questions when they are integrating it into their engine, so there is some cost in taking care of this support. The business model has to take care of this support cost so there is some head scratching to be done before we can happily announce an indie program.

We’ve tried out a few models so far and we have a few indies using us already so if you are an indie and would like to try out Umbra 3 for your game, don’t hesitate to get in touch.

TP: Yeah, definitely talk to us. The industry is changing so fast that one has to adapt and as a pretty small company ourselves, we are pretty nimble about these things. But, of course, we have to get paid too.

How do Umbra’s licensing costs break down?
SL: Our licensing model is pretty typical for a triple-A middleware, but obviously one has to adjust to the times we are in. We have a standard license that includes binary SDK and support and upgrades for a year at a time. Bigger studios typically choose our premium license, which includes the source code and elevated service level. We also do on-site work to help out in integrations and optimisations, plus co-development on features.

Games engines themselves have become even more powerful and often contain features that could disrupt things for tools specialists such as yourself. For instance, Unity contains its own occlusion culling tech. Why should developers chose Umbra if they already have access to such a tool in their engine or SDK?
TP: Well the Occlusion Culling technology in Unity is developed by us at Umbra. We work closely with Unity’s CTO Joachim Ante and their whole team on that for years. Unity 4.3 actually has seriously improved Umbra 3 technology in the Pro version.

Look, even the most hardcore of hardcore, do-it-all-yourself programmers (and we have a lot of those in Finland) are talking to us and using our tech. There’s just no point in doing everything yourself these days. If somebody’s specialised into developing something and does it well, as I like to think we do, there’s no reason not to at least try out the technology.

You have developers using the Unreal Engine and you can buy a ton of plug-in middleware for that, like Substance for textures or Umbra 3 for visibility.

A game engine cannot do everything perfectly or in an optimised fashion, because every developer has different needs and an engine has to focus on something. Hence you got middleware like ours available in engines, so that you have access to that performance boost or extra features if you need them.

What are the latest features that you’ve added to your game development tool?
Lot of next-gen related performance optimisations but our shadow caster culling feature came online a while ago. It basically allows you render shadow much faster. We are seeing our clients starting to use this feature on next-gen consoles now and it’s quite powerful and saves a lot of rendering time.

What are your plans for the future? How is Umbra 3 going to evolve over the course of this generation?
Umbra is now at version 3.3 so it’s quite mature, but there’s always optimising to be done. The thing right now is that next-gen games are almost “brute forced” onto new consoles using tech like ours without much concern for memory use. But that’ll change in the next few years as developers actually have time to develop new code for the PS4 and Xbox One.

There’s a few new features on the roadmap, such as adding in more features for enabling easier use of dynamic elements in the game worlds.

We are also spending a lot of time on Unreal Engine 4. In the long run, we’ll take a look at the GPGPU compute stuff on the new consoles.

Read more about Umbra Software here. And for more on graphics in next-gen games, check out our feature on photorealism