New Tech At E3 – What It Means For Augmented Reality and the NUI
There’s a lot of talk about the new technologies presented by Sony, Nintendo, and Microsoft at the annual Electronic Entertainment Expo in Los Angeles and how they will effect console gaming, but what will be their effect on Augmented Reality and the Natural User Interface, in general? Having spend the last two days at E3 testing out the Move, 3DS, and Kinect (when not busy taking pictures with the, now resurgent, booth girls), I’ll go through the details of the three offerings and what future they might portent.
Sony Move
The new Sony controller, which uses a combination of accelerometer and computer vision input, is touted as providing an unparalleled level of control for the user. The pitch behind the Move, which is very similar in design to the WiiMote, is that the combined input accurately replicates the user’s full range of motion. Sony has even taken to poking fun at the Wii, in this aspect, with their recent ads.
In action, the move doesn’t seem to perform that much more accurately than a WiiMote with the Motion Plus adapter, with the only possible exceptions being that it tracks Z distance (how far the user is from the screen) when the controller is pointed away from the camera and that one player regularly uses two controllers at the same time. Most of the games feel like Wii games too (sports games, in particular), but one cool feature is that the Move is being used in the only ‘traditional” Augmented Reality experience I saw at E3, EyePet.
With EyePet, a digital creature is added to a displayed camera view, allowing the user to interact with it by turning the Move controller into various items or through motion capture “petting.” While this is probably the highest budget AR execution ever put forth (the level of detail shows it), I found the overall experience to be less than compelling, if not more than a little gimmicky. On the bright side, it shows huge potential for using the camera view with the Move for more in-depth AR experiences.
Nintendo 3DS
Not to be outdone this year, Nintendo presented its new version of the DS, giving the user 3D gaming without glasses (on top of new Zelda and Metroid games for Wii). Outside of the 3D display, the 3DS boasts dual back-facing cameras for stereoscopic vision and a faster processor than previous models. Nintendo hasn’t released specs on the processor, but, with the graphics being at Wii level, it wouldn’t be a stretch to assume that they’re taking advantage of either the new 1Ghz ARM chips coming out now or a subsequent cheaper price on the 833 Mhz chip set used in the Wii.
The stereoscopic cameras and faster processor adds up to some really interesting computer vision possibilities, such as calculating the 3D space in front of the user in real time. Since DS applications don’t have the burden of running on top of an extensive operating systems like iOS or Android, the chances of having enough processing power to perform computer vision effectively are higher on the 3DS than any other major mobile device right now.
Personally, I wouldn’t want to use the 3D display with it that much, though. After several minutes of watching trailers and playing demos on the device, my friends and I all had a small amount of dizziness and nausea, reminiscent of the Virtual Boy. I have a feeling that we’ll be hearing about these issues in the press, when kids play the 3D games for hours on end even though there’s a slider on the side to throttle the amount of the effect.
Microsoft Kinect
Project Natal finally has a release name and, no matter how lame you think the new monicker is, the Kinect proved to have the best hype to reality ratio of all this year’s technologies – that is as long as you don’t count the somewhat over-reaching promo video from last year. Ubisoft had a fitness game, called Your Shape, that showed a model of the user from the device’s raw input, and while it was pretty good at rigging the bone structure of the user, it was “blobby” looking enough to show Microsoft’s reported cut backs on functionality to make the Kinect’s price point (the screenshots on XBox.com look nothing like the actual display of the user’s body, shown below).
The response time to movement had about a half second lag on all the games, so it can be assumed that it’s inherent to the device and not caused by software. That being said, the body recognition accuracy was excellent, and I was really impressed with how well it picked up multiple players. While most of the games felt like blatant attempts to cut into the Wii’s casual gaming market, Big Park’s Kinect Joy Ride and Harmonix’s Dance Central felt like they were taking full advantage of the Kinect’s abilities. Overall,the Kinect will most likely prove to be the NUI’s best friend, as it will get the average user more comfortable with controlling digital object through gestures and body recognition.
Summary
The new input mechanisms presented by the three top dogs at E3 this year go to show that Augmented Reality will eventually be encapsulated within the greater genre of the Natural User Interface. As players become more accustomed to using the real world as their playfield and their bodies as the controller, a generation will be bred ready to jump into “Minority Report” interaction and “Terminator Vision” interfaces. Truly, there is as much happening for Augmented Reality at gaming conventions as there is at AR specific functions. For those in the AR development community focused on innovation through the invention of new computer vision engines, it’s a wonderful and exciting thing.
Follow me on Twitter: @blakecallens