The Fascinating History and Evolution of Extended Reality (XR)

Extended Reality (XR) – an amalgamation of Virtual Reality (VR), Augmented Reality (AR) and Mixed Reality (MR) – is a fairly new development, right? Wrong, actually. The notion of 3D reality is nothing new, and can actually be traced back to the 1800’s.

VR can be traced back to work carried out by Sir Charles Wheatstone – he of the Wheatstone bridge, which is used to measure an unknown electrical resistance – in developing “stereopsis”, otherwise known as “binocular vision”. This was the idea that the brain can take separate images from each eye and combine them to create a 3D image. His work on this led directly to the creation of stereoscopes that did just that and brought depth to viewed images. Slip on a VR headset and you’ll be looking at a stereoscopic image.

Things stayed pretty stable in the world of viewed images until the 1950’s when noted cinematographer Morton Heilig developed the all-enveloping Sensorama, which many regarded as being the first real stab at a VR experience. Sensorama was essentially a movie booth that combined 3D, colour stereoscopic video, with audio, smells, and a vibrating chair to immerse the viewer in the movie, and melding all of the experiences together. Unfortunately, it didn’t really deliver the whole package very well, and was quite expensive, so didn’t catch on, but work done by Heilig went on to form the basis of many of today’s multi-experience cinemas.

But Heilig’s work also caught the attention of engineers at the Philadelphia Storage Battery Company (Philco) and the real work on VR as credible systems began. Their goal was to take the stereoscopic idea and place it in a tracking system for the military, giving them the ability to scope out enemy units in real time and in 3D. While only partially successful, the project gave rise to what could legitimately be called the first VR headset.

It was around this time that Ivan Sutherland, an early computer scientist, published a paper called ‘The Ultimate Display’ which outlined a computer-generated system in which the user would not be able to tell the difference between it and real life. This was seen as the first real stab at what would eventually become recognised as a virtual world. Sutherland’s paper started to get people thinking about what could be achieved. By 1968, Sutherland had developed a system called ‘Sword of Damocles’ in which a worn headset enhanced the view of the user with basic computer-generated graphics – this was the first real move towards augmented reality systems.

The progress of XR devices was hampered by both hardware and software and the fact that virtual devices required a particular level of ability amongst both areas to be able to operate at any kind of believable – or usable – level. With this in mind, XR systems had to wait for computer systems to catch up with the intent before they could progress much further. 

By the 1980’s computer systems were beginning to catch up with the aspirations of XR, and computer scientist Jaron Lanier set up VR-orientated business VPL Research Inc, which set about developing the first real VR headset. In fact, it was Lanier who established the term “virtual reality” in 1987, while the term “augmented reality” was first used by Boeing researcher Tom Caudell in 1990. Things started to heat up with the introduction of the VR arcade machines in the early 1990s. Devices like the SEGA VR-1 motion simulator became commonplace, and as hardware and software became more capable, by the mid-90s, affordable VR headsets like the CyberMaxx and the Nintendo Virtual Boy were becoming available for home use.

The scope of VR systems was growing, but they remained tethered to computer systems and used the power of those to deliver the increasingly vibrant graphics and audio. Then, in 2010, Palmer Luckey created the prototype for the Oculus Rift VR headset, which was a roadmap for the future. While this device still used computer processing power, the revolutionary headset reignited interest in VR from many major players in the field of technology. Luckey’s company – Oculus VR – was acquired by Facebook in 2014 for around $2 billion, and the company’s engineering boffins set about making something that would truly bring VR to the masses.

Also, around 2014, HTC Vive was being developed by both HTC and Valve, and was the first device to use ‘room scale’ technology via a series of on-board cameras, allowing the user to move in 3D space. The Vive was released in 2016 and became the standard for the technology.

Augmented Reality was also starting to attract the attention of developers and these subtly different devices – which overlaid virtual content on the real world – started to appear. Google Glass and Microsoft HoloLens are notable devices, but AR has been much more of a success on mobile phone applications with everyday apps like Pokémon Go and Google Sky – which superimposes stars, planets and constellations on the night sky – being very popular. AR is much more aimed at industry rather than gaming applications, and promises to be a powerful force in all manner of serious fields.

Back to VR and the release of the Oculus Quest 2 has shown that the power of VR can be successfully captured in untethered headsets, and these have become the standard that other developers need to match and overtake.  

XR has come a long way in a short time, and with technology getting better all the time, it is becoming a very exciting time for this kind of technology.