Device-dependent AR Tools: The major strength of Google Tango

I assume almost everyone who reads the newspaper, watches television or browses the internet has heard about Augmented Reality (AR) and Virtual Reality (VR). Frankly speaking, these are not new technologies and certainly not just another buzzword that will phase out. AR and VR have been around for at least a decade, slowly gaining maturity and momentum through the years.

AR in particular has already shown a lot of promise: imagine directions getting overlayed on the pathway as you walk around a store, showing you where to find the product you are searching for, or additional information popping up from products on the shelf. AR is considered a technology that is the next game changer and this is evident with major players like Google (Tango, Earth VR), Microsoft (Hololens), Facebook (Camera Effects) and Samsung (Bixby) are all trying to build a share in this lucrative market.

In a nutshell: how does AR work?

Augmented Reality works by overlaying a video or series of images over the view port. This can be your mobile screen, special glasses like the ones from Microsoft and Google or even Snapchat – yes, really, the popular messaging app not only offers colorful filters for pictures and videos, but has recently announced the feature “New World Lense” to push into AR. The basic idea behind AR is simple: By using computer vision the application sees and understands the surrounding, and then augments what is seen by overlaying it with computer generated objects. These objects can be anything from direction arrows, information panels, video walls to Pokemon (I bet you remember them).

Tools of the trade: which AR tools are currently available?

There are numerous tools and platforms available to start an AR project, but they basically fall into two categories: device independent tools, like Vuforia, ARToolKit, EasyAR, Facebook Camera Effect etc., and device dependent, like Hololens SDK or Google Tango SDK.

The basic philosophy of the device independent AR tools is: identify and augment. Vuforia and ARToolKit work on the concept of target, i.e by using the device’s camera they can identify objects in the real world. This can be an image, a QR code or even 3D objects like a mug or a ball. Once the desired object is detected, you can augment it with additional information or objects. This approach can be used for quite a few applications, for example you can print a 2D view of a building in a magazine or a book and when viewed with an AR device, the app can render a 3D view of the same building on top of the magazine. But this approach limits the way the 3D objects can interact with the real world objects. For example, when only using computer vision it is extremely difficult to roll a 3D ball on a real world table.

In contrast, the device specific platforms have a dependency of additional sensors and are built specifically for a device. These platforms offer specialized features, mainly depth perception and motion tracking.

The benefits of Google Tango

Google Tango is a platform that also uses computer vision, but  gives devices the ability to understand their position relative to the world around them, enabling them to detect floors, walls or other objects. For this, Tango uses three core technologies: Motion Tracking, Area Learning and Depth Perception. There are a couple of mobile devices available with these sensors.

Motion Tracking means that a Tango device can track its own movement and orientation through 3D space. Walk around with a device and move it forward, backward, up or down, or tilt it in any direction and it can tell you where it is and which way it is facing. It is similar to how a computer mouse works, but instead of moving around on a flat surface, the world is your mousepad.

Human beings learn to recognize where they are in an environment by noticing the features around them: a doorway, a staircase, the way to the nearest restroom. Tango gives your mobile device the same ability. With Motion Tracking alone, the device sees the visual features of the area it is moving through but doesn’t remember them. With Area Learning turned on, the device not only remembers what it sees, it can also save and recall that information. When you enter a previously saved area, the device uses a process called “localization” to recognize where you are in that area. This feature opens up a wide range of creative applications. The device also uses Area Learning to improve the accuracy of Motion Tracking.

With Depth Perception your device can understand the shape of your surroundings. This lets you create "augmented reality" where virtual objects not only appear to be a part of your actual environment they can also interact with that environment. One example: You create a virtual character that jumps onto and then skitters across a real-world table top. With Depth Perception in place, the application now has the possibility to provide a much more natural placement of objects, detection of planes and surfaces in the observed world.

Where Tango excels

The major strength of tango is Area Learning, which gives it ability to remember a place and improve it over time. This means you can add objects in the real world and the device will remember the position where they are and once you know the relative position of the device from those objects - bingo! With this technology you can create an indoor navigation, which is accurate up to inches (with some additional mesh). Another application you can think of is a virtual room with furniture placed inside. Or even a game where danger is lurking behind every corner, literally!

Tango applications are simply android applications, so the deployment is also the same: simply deploy your individual Tango AR program to the Google Play store and it is available to millions of users. There just remains one hitch: Only devices which support Tango can install it.