Optimising XR Apps for Performance: Best Practices for Developers

VR, as the forerunner to first AR and now XR, was initially dismissed as something that would only really be of interest in gaming circles but quickly developed into an essential system that can be used in almost any industry.  

From healthcare to manufacturing and banking to education, XR has proven its worth. But it has become a victim of its own success; XR applications are now expected to be very powerful, utilising the full abilities of the current range of XR headsets to deliver high-quality content. But this is becoming increasingly taxing on system resources, and it’s not going to get any easier as apps become more complex and power-hungry.

In this article, we will consider the best practices that developers can adopt to get the best possible performance out of their developed apps and pass on the positivity to consumers. 

Performance Problems

Our XR devices are now capable of doing more and handling increasingly complex content. Rather than simple gaming systems, we have found that XR is the perfect medium to augment almost any industry. Banking, education, healthcare, manufacturing, service, STEM, and, of course, entertainment have all become enormous markets for XR and require complicated apps to make the most of the potential. However, the complexity of graphics and the increasing demands on XR systems have created a real challenge. 

Developers must now design and customise their apps to maximise system resources. But as consumers expect more, those developers are having to work harder to ensure that their products meet the expectations. This necessitates designing apps optimally to enhance performance, but what are the most effective methods to achieve this?

Optimisation for All

Optimising an app for XR requires considering the performance aspects from the very beginning of its development. Designing for maximum performance should be at the top of the designer’s list, alongside making the best possible user experience, but these become a trade-off. The better the delivered graphics, generally, the slower the performance.

But there are several ways that the designer can protect themselves against performance drains and still have a good-looking product at the end of it. Let’s take a look at some of the main methods of optimising systems for graphics and performance.

The reduction of polygon counts and draw calls is an obvious area. The optimisation of textures and lighting; the limitations of animations; and the implementation of a single-pass rendering mode are some of the significant strategies that are typically employed for optimisation.  

However, the designer must carefully consider the implications of using these strategies, as they may not always be appropriate for a quality product. Additionally, testing the application on launch-targeted devices is crucial for accurate benchmarking and final optimisation. This obviously becomes much more of a task if several high-end devices are seen as the main launch platforms, but that is what planning is all about.

Some of the key steps that you could take to help optimise your apps include the following:

• Assess system usage. Tools such as the Unity Profiler, the Meta Quest Runtime Optimizer, and the Unreal Engine profiler are ideal for understanding how much system resources – that means GPU, CPU, and RAM – are being used simply to support the app. These tools can also be used to identify potential bottlenecks within the target hardware and ensure a smooth operation on these.
• Simplify structures. Reduce the number of draw calls and the amount of memory that is used by combining textures into a single structure and using fewer materials. When it is possible, use formats that are compressed for textures. You may also help minimise the strain on the graphics system by reducing the number of triangles and vertices in your 3D models. This will assist in reducing the demand placed on the graphics systems.
• Pay attention to your render pipeline. Adjusting the parameters in your render pipeline asset to achieve a balance between performance and graphics is one way to optimise XR applications with the render pipeline. This can be accomplished by removing the high dynamic range (HDR) and complex effects, and you can also consider utilising capabilities such as the SRP batcher to get improved performance with dynamic objects. Be sure that you are utilising the appropriate pipeline for the device you are targeting, and disable any elements that are not necessary to cut down on system overheads, such as volumetric lights or contact shadows. 

Pay Attention to the Animation Pipeline

Additionally, you should focus on the animation pipeline as a means of increasing system flow. The animation pipeline includes the complete application workflow, from the creation of 3D objects and characters to rigging, animating, blending, editing, rendering, and presenting the final animated output. This pipeline has numerous optimisation opportunities throughout the whole process, from conception to execution, aimed at minimising the resources required for animation. 

When people use a VR or AR headset, they expect interactions and motions in the virtual or augmented world to happen as soon as practical, with little lag time between input and response. No one likes lag in their apps and games. Most modern devices don’t have particularly powerful processors. Smartphone processors, for instance, enable many current AR apps and VR headsets that don’t need a full-blown computer to work, just the essential components and subsystems. Compared to regular computing devices, these processors feature slower clock speeds, fewer CPU cores, and less dedicated graphics power but are still capable of running impressive graphics.

One of the greatest challenges facing XR developers is producing responsive and detailed animations that are on par with the quality of flatscreen games and media. Without any kind of optimisation, even simple character movements or actions in virtual worlds might take up a significant amount of processing time. Current XR platforms are under a lot of strain due to complex interactions between numerous animated characters or objects, and future advancements in XR hardware won’t make the situation any better.

When building 3D characters and environments for XR, the designer needs to consider model complexity and polygon count. Every new triangle that must be skinned, animated, and rendered increases resource needs. Similarly, character riggers must weigh the flexibility and range of motion provided by a rig against its complexity and memory footprint. Optimised skinning methods are also helpful for lowering computation costs. 

Techniques such as motion matching, machine learning, and automatic animation retargeting can produce animations that require substantially less manual labour than traditional keyframe animations. Such produced animations must still be reviewed to assure their quality before release. Caching and compression methods can then store up to three-quarters less animation data while preserving overall fidelity. Designers are now paying much more attention to the specifics of rigging and animation and pinpoint a few specific areas to pay attention to, including:

• Use a selection of skinning methods. Dual quaternion skinning (DQS) yields superior quality outcomes by maintaining rotational integrity; however, it is more intricate and computationally demanding. DQS in XR development uses skeletal animation on a mesh to prevent artefacts, such as the “candy-wrapper effect,” prevalent in conventional techniques, by blending transformations with greater precision to maintain volume. Linear blend skinning is the most straightforward and expedient technique.
• Manual motion editing. With the use of tools, animators can manually position figures, modify timing curves, go frame-by-frame through animations, and add post-processing filters. Furthermore, very realistic animations are made possible by real-time motion capture of human actors, where data is instantly mapped onto a 3D character rig.
• Pay attention to layer complexity. When motions in lower priority layers overlap temporally, certain animation layers—such as facial expressions—may be given a greater priority than others and take precedence over them. A wider range of expressive animations may result from this.  Additionally, individual components can be reused across several full-body animations thanks to a set of modular animation layers and blend shapes, which increases overall efficiency.

As XR apps become increasingly complex, designers are having to consider new and novel ways of making the most of system power and optimising for performance. Users expect their apps to run smoothly, without latency or glitches, and to deliver memorable experiences. This expectation requires developers to consider the structure of the animations and how they are delivered on every target device.

If you need help with your XR development, why not come and chat to us at Unity Developers and see how we can help you?