Google’s Tango smartphone uses kinect-like sensor

Google has just unveiled Project Tango, a smartphone with built-in 3D computer vision technology. Think of it as a smartphone with built-in Kinect functionality — but rather than enabling Leap-like gesture control, the computer vision tech is used to create a full 3D map of your current environment. At its most basic, you might use Tango to create a map of your house, or a 3D model of your favorite antique vase or your motorbike. Its advanced uses, though, are what make me truly dizzy with excitement. We are looking at a device that could, in one fell swoop, revolutionize indoor navigation, home improvement, shopping, gaming, and more. How? Read on. Project Tango is a 5-inch Android smartphone developed by Google’s new Advanced Technology and Projects (ATAP) group. ATAP used to be part of Motorola Mobility, but Google decided to keep the division, rather than include it in the recent acquisition by Lenovo. In addition to all the usual cameras and sensors, Tango has a depth sensor, a motion tracking camera, and two computer vision coprocessors, called the Myriad 1, from Movidius (a mobile computer vision startup). These two additional sensors and processors constantly scan your environment, allowing the phone to “make over a quarter million 3D measurements every second, updating its position and orientation in real-time, combining that data into a single 3D model of the space around you.” If you’ve ever seen the raw output from Kinect, the models/depth maps generated by Tango are very similar. Now, Google is trying to work out what you can actually do with all that data. The most obvious use-case is indoor positioning — currently, there is no easy way to use your smartphone to navigate inside a large building. With Tango, your phone could quite easily compare your current location to a previously generated internal map, and tell you exactly where you are — and then direct you to exactly where you want to go, down to the exact shelf location for a product. The same data could also be used to help visually impaired people, by providing audio cues, vibrations if you walk near an obstacle, etc. The other obvious use is gaming. Imagine something like Foursquare, but where each location has an accurate 3D map created by Tango. A mobile device with such powerful functionality is likely to trigger the creation of whole new game genres, though, not just “Foursquare… but with real maps!” It is these new genres and domains that I’m most interested in. Indoor navigation is cool, but there are plenty of other technologies in that area that are much closer to commercialization. Tango might allow you to make a map of your entire house, and then use that map to divine the exact measurements of every wall and piece of furniture within. Then, when you go to IKEA, you might simply say to your phone, “Google, how long is my bedroom wall?” Or, perhaps, when creating a new Ebay or Craigslist listing, Tango could provide a 3D photo of the object and provide all of the relevant dimensions. Again, with technology as powerful as mobile computer vision, it’s not so much about augmenting existing smartphone behavior, but more about the new and exciting apps that will emerge. For now, Tango is a fairly bulky prototype (it’s about as thick as two smartphones). If you have an interesting idea that you’d like to develop, Google has set aside 200 prototype phones for developers. There’s no word on whether the phone will be commercialized, and also very little technical data about Movidius’s computer vision system. One of the issues preventing broader adoption is likely to be power consumption — computer vision, both in terms of sensing and computation, is not cheap. The ultimate goal, I’m sure, is to squeeze Tango’s computer vision technology into Google Glass. Imagine if Google had a 3D map of the entire world, both inside and out, above ground and below. Would that be utopian or dystopian?