The Future of Augmented Reality

There's this cool new technology being developed called Augmented Reality (AR). How it works is there is a camera (such as the camera of a mobile phone or a tablet computer) which is recording some real-life scene. And, in this scene, is an object with a patter on it which an app viewing the live feed is able to recognize. Once it does that, the app attempts to guess the angle of the object it is looking at. This object could be something like a cube having six unique sides, a QR code, or even an everyday picture.

Then, the app superimposes a graphic onto the live feed (which is displayed to the user on their device's screen). This object matches the perspective of the real-life object, so that it appears to exist in the real world. The user can then move the object around and see the imposed object following it. That is pretty much all that AR can currently do. So, while it is a cool little parlor trick, it doesn't serve too much purpose.

With AR being so new, there are a lot of limitations which prevents it from being useful. However, Google is working on their special glasses (Google Glass), which provide a HUD for the person wearing them, by superimposing an image over the persons vision. This is where AR is going to start to shine. If you can see an object in front of you, without requiring any intervention, it is a lot more useful than pulling out your phone and running an app.

There are some more limitations which need to be considered. As one moves their eyes around, the superimposed image on the glass will become offset. This doesn't matter too much with a normal HUD displaying textual information or graphs, but when the image is supposed to align with what the user sees, it will be an issue. This can be solved by turning one's head and keeping their eyes still, but that isn't very convenient. It can also be fixed with a second camera which tracks eye movement and corrects the image accordingly, however that requires more CPU overhead which can cause more image lag, causing the realism to fade.

Another limitation is that only a finite number of patterns can be scanned for at any given time (if using pictures to trigger the image). This can be corrected by a few different things. One thing could be a GPS based library of AR objects. So, if you have an un-moving object which you would like people to be able to see, you would simply upload the triggering imagery as well as the coordinates of the object, and people wearing glasses would only need to be within a certain radius before the glasses are actively looking for said object. A second solution could be data encoded into QR codes, which the glasses could read as a cue, since only the basic QR pattern is being detected instead of every single one (the four big B&W dots that is).

Now, for these AR objects to be truly enjoyed by the masses, everybody with glasses would need to see the same thing at the same time. For example, if I'm looking at this cube and seeing a globe rotated at X degrees so that the USA is pointed north, my friend who is standing to the south of me and looking at the same object should be able to see Asia. This gives off the illusion that these AR objects truly do exist in the real world.

Another important thing for these to work would be a common AR standard agree'd to by different vendors, as well as the AR viewing apps to be built-in to the devices (if Google glasses takes off, I guarantee we'll start seeing iGlasses). The QR code, which was a pretty cool idea IMO, has failed. In order to use a QR code, they need to first download an app, then run the app, then point their camera at the code. If this app were built-in to the OS, and was always running, you would see a LOT more QR codes in the world.

Once hardware is really advanced, having super-accurate GPS, compass, and motion sensors, we won't even need a physical object to overlay a superimposed image over. Imagine going into a field with a bunch of friends, everyone opting to play the same game, and a dragon appears and begins flying around. If the glasses know people's exact locations and angles, it can impose the dragon image in the right location with the right perspective and distance for everyone playing the game.

This is all only the beginning, of course. One day hundreds of years from now we'll have chips in our head with AR running for everybody, and we'll have more realistic features like the ability to touch and feel and smell these imposed objects. But, that story would better be told by George Orwell.

Anyway, I do feel AR will be a big part of an exciting future in a few years, but for now, it is very new and current hardware can't keep up. I don't even know if Google glasses will pick up, it could be that the world isn't ready. But one day some sort of convenient AR integrated hardware will take off.

Thomas has contributed to dozens of enterprise Node.js services and has worked for a company dedicated to securing Node.js. He has spoken at several conferences on Node.js and JavaScript and is an O'Reilly published author.