Augmented Reality in Robotics: How AR Enhances Robots

We are on the cusp of great things. We have woken up to the notion of climate change and are now actively starting to do something about it, and our technology is fast coming up with solutions to problems that just a few years ago, we would have thought insurmountable. We are changing the way that we live our lives and are working in an increasingly family friendly way. If we could just get rid of the notion of money, and it being everything in life, then socially and philosophically, we would be heading to a much better place in our lives. If we can hold our nerve and not muck it all up, then this is the time that we will look back on in the future and say “that was when it all changed”.

We are now used to technology taking leaps forward, but some technological steps have a far more profound impact than others, and it is those greater advances that define where we are as a race and help carve our future. Electrically-warmed toilet seats – though desirable in the depths of winter – are not ground-breaking; robotics and augmented reality are, and when you join the two together, you really have something life-changing.

We have become familiar with robotics and have enjoyed the fruits of fairly simply robotised systems for a couple of decades now. Driven by the rise in micro-electronics and wireless connectivity, we can now make smaller devices do more and increasingly complex tasks. Robot vacuum cleaners and lawn mowers quietly trundle round our houses and gardens and it’s now possible to buy Spot, the robotic dog, so that you can forget to take that for walks too. In forward-thinking countries like Taiwan, robotic factories free people to do more leisurely things, and in hundreds of Amazon distribution centers around the world, automated systems pick the wrong item for delivery for you, taking the place of slower, human operators doing the same thing. Robots have gone from being quirky toys to useful parts of society, and with the ability to rove other planets, they actually do more than humans ever have.

The other impactful technological leap forward has been with VR and AR, and the ability to project people into different environments without them having to even leave their armchair. Also a product of micro-electronic devices and hyper-speed internet connections, XR gives us the ability to become one with dangerous or fantastical locations, and puts us in places that we could normally only dream of. AR gives us an overlay to help understand systems or interact with all manner of equipment. VR can project us into the beautiful or the horrifying, depending upon our mood, and is expected to overtake television as a means of entertainment within the next ten years. The VR global gaming market alone was valued at $11.5 billion in 2019, and is predicted to grow at an annual rate of 30 percent per year from 2020 to 2027. Beyond that, the graph heads towards stratospheric regions. The message is simple; ditch the TV and get ahead of the curve with a VR headset.

But supposing that you took these to pieces of technology and married them together. What would happen then? Well, two things, actually.

On a basic level, like all design functions, the development of highly complex systems such as today’s robots is potentially fraught with problems. A number of people have to work together to ensure that the whole system works; mechanical, electrical, systems and development engineers have to pool their resources. This kind of development has to be carried out in parallel so that all of the different aspects complement each other rather than being bolt-on affairs that don’t integrate properly. Working in an AR environment though, all of these specialists can work together to create a coherent design where everything fits as it is supposed to and works as expected. 

The manufacture of advanced systems is a big issue. Unlike other complex assemblies such as vehicles, the production of robotic systems is a much broader mix of mechanical and electronic systems in a smaller envelope and that means manufacturing in a far more close-knit way. Smaller systems also mean that everything needs to be packaged neatly together to make small, agile devices, and that means a lot of collaboration on prototyping to make sure that it is right long before the Production line starts up. In recognition of this, Professor Karthik Ramani of Purdue University, has taken up training engineers and developers in the art of manufacturing in a VR world, and helping them prepare for intense manufacturing practices for robotic systems.

But what else can the collision of robotics and VR do for us?

Robotics has reached the point where humanoid-based systems can walk with authenticity, and we know that we can connect via high-speed Wi-Fi, and wheeled units can trundle about quite happily. Collision avoidance systems are excellent, and given decent batteries or the ability to periodically quickly power up from mains connections, and robots have a strong future. Allow us to connect to their eyes and systems via an AR headset and we have our own mobile and dextrous buddy to carry out our bidding.

We can send robots to places that would be highly dangerous for us to visit – anywhere outside the atmosphere, deep into the oceans, or into the core of a nuclear reactor – and they can do work with far more finesse than many human fingers. If work means repetitive tasks – and it does for many – then it just means sending your robot to do it for you, leaving you to look in every now and then to make sure that it’s still doing the right thing. It’s never going to slide off for a crafty smoke or ad-hoc tea break so a virtual boss robot is not likely to be standing over your worker one.

Of course, the real choice comes in what we choose to do with the new freedoms afforded us. We could spend the extra time expanding our minds and becoming better people, or we could join together for the sake of the planet and reverse the decline. Alternatively, we could float around in little chairs like the folk from WALL-E, while watching the VR world through the eyes of our robot helpers. I think I’ll go for the last one.