There’s a lot of talk about the new technologies presented by Sony, Nintendo, and Microsoft at the annual Electronic Entertainment Expo in Los Angeles and how they will effect console gaming, but what will be their effect on Augmented Reality and the Natural User Interface, in general? Having spend the last two days at E3 testing out the Move, 3DS, and Kinect (when not busy taking pictures with the, now resurgent, booth girls), I’ll go through the details of the three offerings and what future they might portent.
Sony Move
The new Sony controller, which uses a combination of accelerometer and computer vision input, is touted as providing an unparalleled level of control for the user. The pitch behind the Move, which is very similar in design to the WiiMote, is that the combined input accurately replicates the user’s full range of motion. Sony has even taken to poking fun at the Wii, in this aspect, with their recent ads.
In action, the move doesn’t seem to perform that much more accurately than a WiiMote with the Motion Plus adapter, with the only possible exceptions being that it tracks Z distance (how far the user is from the screen) when the controller is pointed away from the camera and that one player regularly uses two controllers at the same time. Most of the games feel like Wii games too (sports games, in particular), but one cool feature is that the Move is being used in the only ‘traditional” Augmented Reality experience I saw at E3, EyePet.
With EyePet, a digital creature is added to a displayed camera view, allowing the user to interact with it by turning the Move controller into various items or through motion capture “petting.” While this is probably the highest budget AR execution ever put forth (the level of detail shows it), I found the overall experience to be less than compelling, if not more than a little gimmicky. On the bright side, it shows huge potential for using the camera view with the Move for more in-depth AR experiences.
Nintendo 3DS
Not to be outdone this year, Nintendo presented its new version of the DS, giving the user 3D gaming without glasses (on top of new Zelda and Metroid games for Wii). Outside of the 3D display, the 3DS boasts dual back-facing cameras for stereoscopic vision and a faster processor than previous models. Nintendo hasn’t released specs on the processor, but, with the graphics being at Wii level, it wouldn’t be a stretch to assume that they’re taking advantage of either the new 1Ghz ARM chips coming out now or a subsequent cheaper price on the 833 Mhz chip set used in the Wii.
The stereoscopic cameras and faster processor adds up to some really interesting computer vision possibilities, such as calculating the 3D space in front of the user in real time. Since DS applications don’t have the burden of running on top of an extensive operating systems like iOS or Android, the chances of having enough processing power to perform computer vision effectively are higher on the 3DS than any other major mobile device right now.
Personally, I wouldn’t want to use the 3D display with it that much, though. After several minutes of watching trailers and playing demos on the device, my friends and I all had a small amount of dizziness and nausea, reminiscent of the Virtual Boy. I have a feeling that we’ll be hearing about these issues in the press, when kids play the 3D games for hours on end even though there’s a slider on the side to throttle the amount of the effect.
Microsoft Kinect
Project Natal finally has a release name and, no matter how lame you think the new monicker is, the Kinect proved to have the best hype to reality ratio of all this year’s technologies – that is as long as you don’t count the somewhat over-reaching promo video from last year. Ubisoft had a fitness game, called Your Shape, that showed a model of the user from the device’s raw input, and while it was pretty good at rigging the bone structure of the user, it was “blobby” looking enough to show Microsoft’s reported cut backs on functionality to make the Kinect’s price point (the screenshots on XBox.com look nothing like the actual display of the user’s body, shown below).
The response time to movement had about a half second lag on all the games, so it can be assumed that it’s inherent to the device and not caused by software. That being said, the body recognition accuracy was excellent, and I was really impressed with how well it picked up multiple players. While most of the games felt like blatant attempts to cut into the Wii’s casual gaming market, Big Park’s Kinect Joy Ride and Harmonix’s Dance Central felt like they were taking full advantage of the Kinect’s abilities. Overall,the Kinect will most likely prove to be the NUI’s best friend, as it will get the average user more comfortable with controlling digital object through gestures and body recognition.
Summary
The new input mechanisms presented by the three top dogs at E3 this year go to show that Augmented Reality will eventually be encapsulated within the greater genre of the Natural User Interface. As players become more accustomed to using the real world as their playfield and their bodies as the controller, a generation will be bred ready to jump into “Minority Report” interaction and “Terminator Vision” interfaces. Truly, there is as much happening for Augmented Reality at gaming conventions as there is at AR specific functions. For those in the AR development community focused on innovation through the invention of new computer vision engines, it’s a wonderful and exciting thing.
In response to Apple’s recent shutting out of anything Flash on the iPhone, iPod, and iPad, Mike Chambers, Adobe’s product manager for the Flash platform, made a statement about Apple mobile development.
“As developers for the iPhone have learned, if you want to develop for the iPhone you have to be prepared for Apple to reject or restrict your development at any time, and for seemingly any reason. . . The primary goal of Flash has always been to enable cross browser, platform and device development. The cool Web game that you build can easily be targeted and deployed to multiple platforms and devices. However, this is the exact opposite of what Apple wants. They want to tie developers down to their platform, and restrict their options to make it difficult for developers to target other platforms.”
In one of the most interesting official statements to come out of Apple in a while, Apple spokesperson Trudy Miller responded,
“Someone has it backwards – it is HTML5, CSS, JavaScript, and H.264 (all supported by the iPhone and iPad) that are open and standard, while Adobe’s Flash is closed and proprietary.”
Let’s take a second to get away from the spin and run through the facts about which side in this argument is more “closed and proprietary” than the other.
Distribution
iPhone: Apple maintains a closed distribution model (the App Store) that is the only way to get applications without breaking the warranty of your device. Apple also maintains a very profitable closed distribution method for music and video, and only allows select partners to deliver media outside of it.
Flash: While the Flash plugin is proprietary, it is available for use through all major web browsers and practically every smart phone, other than the iPhone, at no cost to the user and without limiting other interactive content platforms. Also, with Flash, the user has open access to many sites that offer free delivery of music and videos.
Development
iPhone: In order to develop for Apple’s mobile devices and distribute to the majority of users, the developer must pay Apple $99 a year for the right to not only submit applications and keep them in the App Store, but just to test them on a device. Compare this to Android, where all of that is 100% free. Also, a developer must sign an extensive terms of service agreement that effectively gives Apple the right to steal their application and release it as their own intellectual property.
Flash: While the Flash CS5 and Flash Builder development environments cost about $500 a piece, Adobe also provides the Flex SDK, which has the exact same coding abilities, completely free. It is relatively easy for a developer to make and distribute Flash applications, including desktop apps through AIR, at no charge and with no approval process or possibility of having it stolen by Adobe.
Other “Closed and Proprietary” Things That Apple Does on Mobile
Apple doesn’t allow fully functional third party apps that “duplicate functionality.”
Apple doesn’t allow apps that “ridicule public figures,” though the 1st Amendment does.
Apple limits how apps can use public data that can be accessed through the browser anyway.
Apple limits how third party Ad systems can send statistics.
Apple limits what languages can be used to make applications. Note that not one of Miller’s supported “open and standard” languages is Object-Oriented and capable of delivering native quality applications, through the browser, like Flash can.
With this list, I’m not trying to say that one side is morally better than the other, but Apple calling out Flash for being “closed and proprietary” is most certainly hypocritical, if not downright ridiculous. Personally, it’s getting annoying to have to regularly break down the spin, and sometimes logical inconsistencies, of an Apple statement to come to an estimate of the real reasoning behind the action (something I must do, because their decisions affect my profession). I can’t help but think that if the real reason was at all benevolent, such as supporting open standards, there would be no need for these types of blatant cover-ups.
I never cease to be amazed at the ferocity with which Apple fanboys will defend the latest iProduct, nor with how the opposition to their blind devotion is presented with an equally uninformed opinion. The arguments seem to regularly break down into two sentiments:
Apple Fanboy: “All you Microsoft/Linux slaves/geeks don’t understand the beauty of Apple’s design. Can’t you see that Steve Jobs is trying to bring us into a digital utopia.”
Opposition: “Can’t you see that Steve Jobs is Big Brother/Satan!?”
Both arguments lack an attempt to analyze the core of the issue at hand, and serve to only further confuse non-techies that don’t belong to either camp. In order to better understand the rational behind Apple fanboys’ side of the argument, let’s go through the corporate marketing strategies that shape their opinions and, if you’re of the other persuasion, the proper way to retort.
The Cult of Mac
The brilliance of Apple’s marketing strategy is that it appeals to one of humanity’s basest emotions, the desire to feel superior to others by tying oneself to a cultural or group identity. Apple loyalists are led to believe that they are going against the grain of society, distinguishing themselves from average people buy purchasing a product that is so perfectly designed, so inherently simple to use, so free of bugs. Others that choose to follow the same path are their brothers and sisters in arms, while those that don’t are slaves to – dare I say it – bland corporate UI designs in a buggy operating system. Like any proselyte, they can’t see that their leader, like all evangelists, exaggerates the shortcomings of any teaching that contradicts his own.
Because of this need to maintain their unwaivering status as a member of the group, and because admission that the latest product is anything less that spectacular would also be admitting that they were wrong about Apple’s (supposed) flawlessness, the fanboy wraps himself in a cocoon of cognitive dissonance. This is best exemplified by comments made by a fanboy on my post, Why I Won’t Be Buying a First Gen iPad.
“I get it – you don’t get it. Which is OK. . . You’ll get it better soon, if you will allow yourself.”
And in a follow up comment. . .
“I’ll be at the coffee shop with my iPad. You’ll have your Ubuntu netbook.”
First, the fanboy asserts that if if have reasons for not being interested in the iPad, it’s not because they’re valid, but because I “don’t get it.” I would be able to get it if I allow myself to see the beauty of his Cupertino oriented point of view. Then he unwittingly admits his true reason for wanting the device: to be seen using it in public so he can be identified as one of the chosen few.
“He’s inventing a new world. History will just follow.”
No wonder they call it the Cult of Mac.
This is not to say that there aren’t others who prefer some of Apple’s products, but don’t follow the company line. I, for one, love the user flow of OSX, especially because it’s a beautiful UI wrapped around a UNIX shell. I’m writing this in Open Office on Ubuntu Linux, though, so my fanboy status is negated by membership in an even smaller faction.
Responding Through Passive Aggressiveness
Trying to counter an Apple fanboy’s opinion is like trying to convince someone that their religion is pointless; it’s just not going to happen. Instead of confronting them head on, consider flanking them with one of these questions:
“I’m having trouble with my iPad, could you help me? I can’t seem to find the button that allows me to access the file system.”
“I was playing Farmville on my friend’s Android phone the other day. When I tried to do it on my iPhone, it didn’t work. Do you know why?”
“My boss left me a message on Google Voice and I need to hear it right now. I’m away from my computer, so is there a way I can do it with my iPhone?”
Perhaps the best way to respond is through self reflection, asking yourself, “Do I, with my own gadget preferences, operate with something of the same cognitive dissonance by rejecting everything Apple?”
Let me pose a question: where would personal computing technology be today if software developers had always been subject to any and all of hardware manufacturers’ approval guidelines and whims?
Imagine a world where:
You are forced to use your desktop or laptop’s bundled web browser, email client, and media applications, because other options “duplicate functionality.” That means no independent solutions like Firefox, Thunderbird, or VLC.
There are no P2P clients, like Transmission or uTorrent, not because the software itself is illegal, but because the consumer might use it for “infringing third party rights.”
Great games, in the traditions of Duke Nukem and Leisure Suit Larry, are unavailable because they contain “objectionable content.”
You can not manually manage files, but can only edit, save, and delete them from within the approved applications that they were created in. You can not download a file from the web and open it.
Software can’t use the rights of parody and satire, granted by the First Amendment, because the hardware manufacturer won’t approve an application that “ridicules public figures.”
Even if the software didn’t break the previous guidelines, manufacturers could deny its release, negating a developer’s months, if not years, of hard work, just because they felt like it.
Developers who write groundbreaking applications have to agree to give the hardware manufacturer the choice of duplicating the application and releasing it as their own intellectual property.
Sounds Orwellian, doesn’t it? This is exactly how the closed mobile development space works right now. And, as the distribution model is gaining popularity among other hardware developers, it’s not much of a stretch to imagine a near future where personal computers have adopted this format (a vision held by Popular Science’s Tom Conlon).
Although the closed development model is overtly damaging to the software industry, developers are flocking to it in droves, disregarding their best interests for a chance at being one of the lucky few to make a killing off of a popular application. At the same time, fully aware of the downside of the process, they are less likely to submit any software that is truly groundbreaking. This leads to a mobile marketplace containing mostly ports of browser based applications, information reference software, minimally functional “one task” applications, and games.
If you doubt that statement, and own one of these devices, write down the number of third party applications you have installed, then list the applications that aren’t games or just clients for things you can do in the device’s web browser (if the device doesn’t have Flash, pretend it does). Here’s my list:
117 Third Party Applications
80 Games (I play only 6 of them regularly)
8 Qualifying Applications, comprised of:
1 Remote Mouse Application
1 Earthquake Warning Application
2 Location Check In Applications
4 Augmented Reality “Lite” Applications
Out of 117 third party applications on my device, only one provides both a practical, utilitarian purpose and something I can’t already do in the browser (the remote mouse). If you think my results are biased, I wrote one of the 109 applications that didn’t make the cut. There are some other truly useful applications that I could purchase for my device, but they’re as much a needle in the hay stack as the one I already have installed. Also, if I find an application that I want, but it wasn’t approved for release by the manufacturer, I can’t get it without breaking my device’s warranty or going through black market-like ad-hoc distribution. How would you feel if a desktop computer, for sale in an electronics store, were hindered by those limitations? Would you still buy it, or would you choose something else?
The best way that the software community can keep closed development from spreading to the desktop, or at least ensure that there will always be an “option B,” is to support open mobile platforms. In my next post, I’ll describe paths towards accomplishing that without forsaking profits.
There’s been a lot of talk lately about HTML5 and how it will completely change the landscape of the web. Folks as high up on the technology food chain as Steve Jobs tout it as the ultimate replacement for Flash. So, in an effort to be a good programmer, I spent some time learning HTML5 (specifically the canvas component) to compare capabilities. I can emphatically say that HTML5 will not be a Flash killer, with the only possible exception being video, and here are the reasons why:
Just try to get a programmer to drop OOP
Flash is run through the ActionScript language, and AS3, the version of ActionScript that’s been around for almost four years, is object oriented. For those not familiar with OOP, it’s by far the most preferred method of coding for experienced software engineers, most of who consider non-OOP languages archaic (probably the nicest thing they would say).
On the other hand, JavaScript, which runs the HTML5 canvas component, is not a fully object oriented language, though in some ways it attempts to mimic OOP. Creating a fully functional Rich Internet Application (other than basic information aggregators) in JavaScript is frustrating at best, and down right futile at worst. Any way you slice it, moving from AS3 to JavaScript is a step down in capability, and no experienced programmer will rush to do it.
I have to manually support different browsers?
A Flash application will run on any operating system in any browser. The only possible issue is the display of system fonts, which can easily be averted by embedding fonts in the application. HTML5, though, brings with it all the same browser compatibility problems that currently plague web designers. A HTML5 application playing sound or video will have to support every codec used by all major browsers. So, instead of having one mp3 for every sound in Flash application, the developer will have to make a mp3, wav, and ogg for every sound that needs to be played and write code to test the browsers supported codecs.
The same goes for video. Websites that want to support video playback will have to encode all their videos in both the h.264 and Ogg Theora codecs. Imagine a site with a video repository the size of Hulu’s deciding to go through every single one of their thousands of h.264 videos and re-encoding them for Ogg, especially when their Flash video player works just fine for them already. YouTube recently launched an HTML5 player, but decided not to support Ogg, so the only browsers it can be viewed in are Safari and Chrome. Beyond the codec issue, the HTML5 video component actually does work very well, and I suspect that we’ll begin to see it used more often.
I’ve also noticed some screen redrawing issues that vary between browsers. When redrawing large bitmaps on the canvas, the area would flicker sporadically, as if the browser couldn’t catch up to the redraw calls. Once again, this is not an issue in Flash, since it takes care of redrawing the stage automatically.
Code obfuscation, anyone?
Code obfuscation is a process by which interpreted code, such as JavaScript, is modified to make it difficult to read without modifying the functionality of the application. It’s used to keep code that has to be public from being stolen. With JavaScript, you have no choice but to make the source code public and anyone with a moderate knowledge of web browsers can easily steal it, so obfuscation is a fairly common practice.
To help illustrate the current issues with code obfuscation and the canvas component, here’s a little Pong game I wrote in HTML5. You’ll only be able to play it if you have a capable browser, such as Firefox, Chrome or Safari.
I attempted to use every online JavaScript obfuscation tool with the source code for that Pong game (only 280 lines long), but not one of them returned me code that worked. I suspect that’s because they’re not able to handle HTML5 calls yet, or perhaps they don’t really work that well in the first place. For the sake of argument, let’s assume that you, a developer, have successfully obfuscated your JavaScript code and have put it online. An inventive hacker could find out which tool you used and reverse the process to get, at least, something close to the original source (encoded variable names might not be restored). While Flash has it’s own obfuscation built in, and there are many decompilers for it, it’s a semi-interpreted language, so deconstructed applications reveal only snippets of the original source.
While I’m all for open source, site security dictates that some code has to be as close to impossible to hack as humanly possible (no code is 100% hack proof). Most secure functions on websites are done on the server to keep hackers from reaching it through the browser, but with Rich Internet Applications that deal with secure information, there has to be an intermediary point where the application sends and receives sensitive data. It’s much easier for a hacker to retrieve that procedure through JavaScript, obfuscated or not, than from a plugin.
Flash to the rescue
The idea behind HTML5 is a solid one. The kind of functionality that Flash delivers is ultimately something that should move out of a plugin and in to the browser, but this isn’t an easy change to make. The main roadblock is the core difference between web and application programming. HTML has never been more than a document format, and through the last fifteen years of mass marketed internet, developers have tacked on external languages to give it greater functionality. In the last few years, Flash has become the most used way to bring native application functionality to an HTML page, and it was able to do it because AS3 is a language of substantial depth, based on OOP principles.
Because of this, I think the best hope for HTML5 is for Adobe to add an “Export To HTML5″ feature to the Flash IDE and Flex SDK. I suspect talk of it is already floating around their headquarters, because, despite what someone said, they are most definitely not lazy.
I used to hate Macs. A work experience with a G3, that seemed to crash every ten minutes, turned me off to anything Apple for years. Everything changed, though, when I got my first Intel based Mac in late 2008. The user interface was flawless, the UNIX shell made it a cinch to do back-end tasks, and I never had to worry about the issues that plagued that old G3. I haven’t bought a PC since (at least not without immediately wiping the hard drive with Linux) and I’m known around work as a hardcore Apple fanboy, but that might be changing really soon.
The vast majority of applications I write are for Flash (written in Flex). I love the ease of Flash development and rarely have any issues with programs I write for it. That’s why I’m particularly troubled with, what seems to be, Steve Jobs’ fevered desire to destroy the platform. If you don’t know what I’m referring to, Jobs reportedly told a group of Apple employees that Adobe, the developers of Flash, are “lazy” and that the platform is “buggy.”
I call shenanigans.
There’s a common myth about Flash circulated by developers not familiar with the platform (also known as “my platform is better than your platform” developers). They say that Flash causes most browser crashes when, in fact, it’s not Flash itself causing most of those crashes, but bad Flash coding by inexperienced developers. To put that into perspective, I’ll pose a rhetorical question. When an iPhone developer makes an app that crashes constantly or leaks so much memory that you have to restart the device, does anyone say that Apple should dump Objective C and Cocoa?
Since making the move to Flex, a while back, I haven’t gotten word of a single one of my applications ever causing a browser to crash. This is after extensive use of these applications, not only by the public, but also by a quality assurance department as well. Flash is a very stable platform when used correctly.
The iPad and Flash
Adobe has had a fully working version of Flash ready for all major mobile devices for a little while now. Other mobile operating systems, like Android and Windows Mobile, are going to use it, so why not iPhone or iPad? Steve Jobs, among others, would have you believe that it’s because the platform isn’t good enough for their devices, but that’s just a cheap cover-up. The real reason is that it would take a massive chunk out of App Store profits.
If you own an iPhone or iPod Touch then you know that the majority of apps for those devices are – and I don’t say this lightly – complete crap. The App Store is overrun with weekend projects by novice developers and rip-off products by companies re-skinning the same worthless app to get more 99-cent sales, and Apple is making a killing off of the suckers that buy them. Don’t believe me? Look at these screen caps from the App Store for latest releases in the Entertainment and Lifestyle categories, taken while I was writing this.
If Flash were allowed on these devices, almost all of those developers would opt to release their apps as “iPhone ready Flash,” raking in all the revenue from dozens of ads for enlargement pills and naked video chat, which they would surely cover the page in. If Apple allowed Flash on iPad and iPhone, App Store profits would drop by a double-digit percentage practically overnight.
Why HTML5 won’t be a Flash killer
Because of this, Apple is publicly throwing their hat in the ring for HTML5, all while completely ignoring some of the glaring problems that come with it. Mind you, HTML5 is a good advancement for the web, but it’s anything but a Flash killer, and here’s why:
1. Video codecs aren’t free. The reason that Flash works with all sorts of video codecs is because Adobe paid for them. When that responsibility is moved to the browser, problems ensue, because not every browser supports all those codecs. Mozilla has chosen Ogg, Apple is with h.264, since that’s what iTunes runs off of, and while Google is also currently with h.264, word is that they might be about to opt for a codec they just gained ownership of. This means that not all HTML5 video players will work in all HTML5 capable browsers. This is a huge headache for developers, since cross-browser is the name of the internet game.
2. HTML5 can’t make the level of games that Flash can, or at least not with anything near the same ease of development. Casual Flash gaming is a huge moneymaker, and no one is ready to drop it anytime soon. Just for fun, ask just about any Flash game developer what he would think about dumping ActionScript to write a game in JavaScript and enjoy watching his head explode.
3. Whatever HTML5 will do, Flash can already do it and then some. As time goes on, and more functionality becomes available with HTML5, Adobe will, through their normal development cycle, give Flash more and more features that HTML5 won’t be able to compete with. Two years from now, it might be that video players and simple rich internet applications are best done in HTML5, but, in order to do the latest things available, you’ll still need Flash.
4. Flash has a dedicated user base. 99% of computers in the modernized world have Flash installed. Safari and Chrome, which make up about 14% of total browser penetration, are the only browsers capable of running YouTube’s HTML5 video player. Things might change for HTML5 penetration, but not soon.
The future of my Apple fanboy status
I’m struggling to remain an Apple fanboy, with their ridiculous practices regarding Flash and closed development. The truth is, while I’m not happy with the direction their mobile division is going, I love OSX to death. I guess, as a developer, I still admire the hard working people at Apple who, behind the scenes, actually do all the heavy lifting that results in great products.
On the other hand, Steve Jobs is starting to look less like an innovator to me and more like the delusional cult leader that so many Windows fanboys characterize him as – blindly saying that technologies that don’t conform to his wishes are buggy and that their developers are lazy. I can’t help but think, at least for a while, that I won’t be drinking his special brand of Kool-Aid.
Oh, and he also reportedly trashed Google because they’re making competitive products. When the Google Tablet mockup shows multitasking and multi-touch, the Chrome OS has open development, and the device will almost assuredly run Flash, it feels to me like nothing more than sour grapes.
I haven’t seen anything hyped this much only be such a disappointment on release since Crystal Pepsi. That’s what the iPad is, the Crystal Pepsi of computing (no John Scully jokes, I promise).
We were all hoping for something to lead us into the next generation of mobile computing, but what we got was a giant iPhone with a couple new features – the kind of features that should be in iPhone OS 4.0 anyway. Steve Jobs opened up the media event by saying, “…netbooks aren’t better at anything,” but I can’t help but be glad that I bought a netbook and put Ubuntu Linux on it last month instead of saving up for the iPad, and here’s why:
My netbook has a faster processor
Most netbooks run on a 1.6Ghz Intel Atom processor, while iPad runs an A4 1Ghz processor. To put that into perspective, the iPhone 3GS runs an 833Mhz processor, so this is not anywhere near a huge jump up in processing for a device four times the size of its predecessor. The A4 processor gives the iPad about 10 hours of battery life, which is impressive, but you’re not going to be able to do the level of processing available in most netbooks.
My netbook has a camera
This is a WTF non-feature for me. Apple had a chance to make the iPad the premier Augmented Reality device of 2010, but instead they’ve left every AR developer scratching their head. It’s not as if there isn’t enough room to put two cameras, one facing each way, in the device. How many more iPads would Apple sell if you could use Skype with it?
My netbook runs Flash
Apple’s continued cognitive dissonance when it comes to Flash on their mobile devices is getting rather annoying to me, and not just because I develop on the platform. Flash pretty much runs the interactive web nowadays, and it’s exclusion makes the iPad a “web-lite” device, unable to run Vimeo, Hulu, and any media embeds on social networking sites and blogs other than YouTube.
Apple has a good reason for not allowing Flash, which, believe it or not, is actually ready for the device – it will hurt AppStore profits significantly. Yet, when 90% of apps in the store are complete crap, who cares if their developers move to Flash. The real moneymaking apps will still be iPad and iPhone native.
My netbook runs a full OS
I have full install of Ubuntu 9.10 running on my Acer Aspire 1, giving my netbook the full functionality of a normal laptop. I can multitask on multiple, switchable workspaces, using fully functional applications, and I can engineer and compile new applications. I can browse through folders and files. I can do full network administration on a device that is taller than the iPad, but smaller in width and height.
I can do everything on my netbook that is capable with the iPad and then some, minus the slick user interface. To be fair, one of my favorite things about Apple products are the interfaces, but my MacBook and iMac back up the beautiful interface with tons of under the hood functionality. The iPad offers less than I can already do between my MacBook and iPhone.
My netbook cost half the price of the cheapest iPad
I bought my netbook at Fry’s Electronics for $250. It has a 160Gb hard drive, while the most expensive iPad model only has 64Gb for $829. The cheapest iPad only has 16Gb, or half of the storage of my iPhone 3GS, and costs $499. This is unacceptable for a ten inch device when you can get a 16Gb Micro SD card on Amazon for less than $50. Also, my netbook has two SD card slots on it, allowing me to expand it’s storage capabilities.
What Improvements would get me to buy the iPad
I have hope for future generations of the iPad if Apple integrates the following:
Front and back facing cameras
An OS that is less like the iPhone and more like OSX
Allow Flash 10.1 on the device
Make a slightly thicker version with a full hard drive and USB and SD ports
Swap out the A4 processor with an Intel chip
Stop hyping the current model like its something more than a big iPhone
I don’t expect these changes anytime soon, making this the biggest opportunity for Microsoft and Google to outdo Apple in years. A Windows, Android, or other Linux kernel iPad clone that has all the features serious computer people are looking for would deal a huge blow to the folks in Cupertino.
There’s been a lot of buzz the last few days about Google’s foray into Augmented Reality territory, but it’s the non-AR (technically) side of their new app, Google Goggles, that steals the show. I downloaded Goggles to the company Android phone today and gave it a test run. Actually, test run is an understatement – I ran around the office taking pictures of everything, amazed by the power of its photo recognition engine. One must assume that Goggles is, at least partly, the outcome of Google’s purchase of Neven Vision, an image recognition startup, back in 2006. It just might wind up being the company’s smartest aquisition, since it’s becoming safe to say that Google will own location AR once they get Goggles analyzing off a real time video feed.
While Goggles can recognize logos and graphics (it even recognized a painting in our office), upon use it’s obvious that it gets most of its power from text recognition – really good text recognition. The only thing it had trouble with were the words “BIG BOY” on my piggy bank, which it read as “BIG SOY” and then “BIO SOY,” both of which yielded interesting search results. As Google refines the engine to work faster and more reliably, though, it could wind up being the basis for true hyper-location AR.
Imagine this – a mobile app gets your relative position and direction from GPS and compass data (which Goggles already does), then behind the scenes, takes a snapshot from the camera. The image is analyzed by the recognition engine and matched with Google Street View images within your vicinity. Based on the scale and angle difference of the Street View image and the snapshot, the application is able to find nearly your exact location and place digital objects accordingly. No more will location AR be vague popups of information, but accurate modification of a real world view. To let up on processing and data transferring, the app could make this call every ten to twenty seconds and still produce results that would put every other location based AR company out of business.
I’m sure that someone at Google has already thought of something like this, so be ready for each successive update to Google Goggles bringing us one step closer to the ultimate vision of Augmented Reality.
For all you folks that don’t have an Android phone, here’s a video of me using Goggles around the office.
There’s been a lot of negative talk about Flash AR executions lately, and it’s deserved. Too many brands are using development houses that have no more understanding of the FLARToolkit than throwing a 3D model on a marker, let alone how to use it for successful product promotion. So, I completely understand why some of my peers, mainly those in the mobile AR realm, are rolling their eyes at the title of this post. Hopefully, I’ll make a good case for my claim, which is that Flash is the best breeding ground for new AR concepts that can then be brought to a wide array of other platforms.
_________________________________________________________________________________________
Mobile, Location based AR is the star of the moment. Many developers in the genre state that it’s what Augmented Reality is all about, but these GPS and compass based applications are only a small fraction of the types of engines being created and even already readily available. This has led to a rift in the AR developer community, causing the term, “True AR,” to be regularly used.
What is True AR? It’s the real time calculation of camera data being fed by the device to create an AR experience. Almost all web AR executions are True AR, although most of them right now are just True Gimmicky AR.
The limitations of GPS and compass units on mobile devices makes current mobile AR apps light on usability. The developer community has coined the melding of True AR with this genre to provide more accuracy, “Hyper-location AR.”
Unfortunately, there’s a problem with Hyper-location AR and any other True AR experience on mobile devices: processing constraints. It takes a whole lot of calculation to process camera data into a virtual construct, and while native applications on desktop computers are handling it well, mobile devices are still a little sketchy. This brings me to my first point:
Flash’s processing restrictions are a good thing for AR
Eye rolls again, I’m sure. Here’s the thing – if an algorithm runs well in Flash, you can be sure that it will run brilliantly on any mobile platform. Flash’s processing restrictions keep it running about the same, or just under, the iPhone, Android, and Symbian devices, when it comes to computation. An algorithm that allows me to track a hundred separate points in an image, per frame, might work out to 125 to 150 points on a mobile device (admittedly a guestimate). Meanwhile, an algorithm working in Java on Android might work a little slower in Objective C on the iPhone, or vice versa. When you start with Flash, you’re always moving up.
Flash is a very easy language to port
Yes, Flash is my favorite platform, so take this with a grain of salt. I didn’t get my start programming in ActionScript, though, and I’m well familiar with c++, Java, and Objective C, which run Symbian, Android, and the iPhone, respectively. Let me use c++ as an example, since it’s the primary language being used to create high-end AR engines.
If you take apart a large c++ library like bazAR, a feature recognition engine, it can be like reading Klingon. That’s because of how c++ variable naming has an effect on the processing of an application. In an engine like bazAR, that’s performing a huge amount of calculations, keeping your variable names as small as possible makes a huge difference in the final product. Problem is, it makes it a pain in the neck to figure out what’s going on when every step isn’t completely commented by the developer (99.9% of engineers barely comment at all).
If you’re a development house that’s created an AR engine in c++, but want it to work on a wide variety of devices, either you’re lucky enough that the creator is familiar with every single mobile platform, or you’re going to have to bring in an outside developer to port it. Having cryptic code, which is almost unavoidable in heavy c++ libraries, adds a good amount of man-hours to that work, which translates to more money spent than necessary.
Why more than necessary? Because Flash variable naming has no determination on processing. I could name a variable public var thisVariableIsReallyCoolAndIThinkYouWillAllLikeIt and it would take the same amount of processing as public var i. While a c++ engine would likely have a function with a dozen single letter variables, that require someone reviewing the code to decipher the calculations just to get a sense of what’s happening at all, a Flash one could have a descriptive name for each variable that makes the function easy to understand. Basically, Flash is the closest to pseudo-code a full-featured language gets. It can be the Rosetta Stone for your AR engines.
I know that some folks are thinking, “But you can just write a wrapper for c++,” but if you do that, you’re adding extra processing in an already processor intensive app. It only makes sense to port to the native language of the device.
Flash has an ease of development not offered by other languages
You might think I’m gushing on Flash again with this, but it’s the absolute truth. ActionScript takes the developer from idea to proof of concept faster than any other language. Many of the back end calculations that need multiple third party libraries for other platforms are built into Flash and are extremely simple to utilize. I’ve had an idea about a body recognition algorithm at 10am and had it running relatively smooth in Flash by noon, ready to show as a new proprietary library. Try that on a mobile platform.
Also, Flash has, arguably, the most vibrant developer community online. The barriers to entry are low – sometimes too low, as is demonstrated by gimmicky AR – but it’s simply the easiest language for a seasoned developer to learn. You can Google the answer to just about any specific Flash programming question imaginable.
Accessibility, Accessibility, Accessibility!
So you’ve created an experimental AR algorithm and you need people to test it for you, break it down, and help you squash any bugs or weaknesses. Let’s say you wrote it for Symbian. How many people in your office, let alone your QA department, have Symbian phones? Translation: potentially thousands of dollars of investment for something that’s experimental.
Now, say that you built it in Flash. Try to name someone at your office that doesn’t have a computer with Flash installed. Sure, you might have to pay for webcams, but they cost a whole lot less than smartphones. Plus it’s ridiculously easier to do a public beta test with Flash, where someone will just navigate to a site, rather than having to manage users downloading a regularly updated app to their phone. Once the kinks are worked out in a Flash build, port it to Symbian or any other device, where your test time and budget are a fraction of what they would have been if you went the other route. That and you have a working demo that doesn’t require a potential client to have a special device.
Full User Interaction
If using GPS and compass input is AR, and using camera data is True AR, then adding motion capture and body recognition to the mix is Really True AR. The algorithms that run body recognition in current AR will be the basis for the so called “Terminator Vision” that tech bloggers love to talk about lately. Nowhere is it easier to construct and test these algorithms for mobile use than in Flash.
Something as simple as drawing vector body outlines for results testing, or parsing bitmap data for motion recognition, could potentially take hours or days with any other platform, but is done in minutes in Flash. Why spend any more time than is necessary when experimenting with one of many hypotheses? I couldn’t tell you how many of my ideas went bust within an hour or two with ActionScript, when it would have taken me a whole day or more to reach the same conclusion in another language.
The most important thing is money
Less money spent, to be exact. Go the Flash route and you can have an AR algorithm ready to be ported to any platform without hassle, free of device specific features, guaranteed to run smoothly, at a fraction of the effort and cost of other methods.
Now, let the text based eye rolling begin in the comments section.
I saw a tweet the other morning stating that “Augmented Reality Goes 3D,” which was a total shock to me, considering that I didn’t wake up in 1999. What shocked me even more was that it was the title of a post on a well known blog.
For a software engineer who spends most of his work day developing AR, I think I handle the hype of this tech pretty well – in that I don’t go in to a psychopathic rage every time I read a new, poorly researched article (Jack Benoff might disagree). So, why do I have such a deep seeded hatred for bad tech reporting? Because writing how an iPhone app you’ve never used is SUPER AWESOME makes people think that AR, in general, is a joke – at least it does when they actually use the app and find that it’s a gimmicky piece of you know what.
Don’t believe me? Here’s a list of some tweets that I picked up just this afternoon:
“I would say most augmented reality examples out there are defintely [sic] useless crap. Without a doubt.”
“Augmented reality blows, just like virtual reality did. Until people perfect the technology it shouldn’t be used in the main stream.”
“Technology is getting ridiculous – I don’t need augmented reality to find my car; I just need generic “normal” reality.”
“Is Augmented Reality overblown? Read this article and try to imagine a real use for #AR that isn’t a fad.”
“Yes, we have been here before RT @PSFK: Is Augmented Reality The Next Second Life?”
And, no, this isn’t just due to Twitter trolling. Here are some posts from popular blogs that say much of the same thing:
So is the blogosphere the only one to blame for this? Not really. Much of the blame is also with the fly by night companies that are so frantic to publish something they can label Augmented Reality solely to attract attention. Their products usually lack polish and reliability. Also, some developers say that their products are a lot more accurate/useful than they actually are (I won’t mention names) and proactively bring about user disillusionment.
I don’t want to come across as only complaining about the AR industry. There is REAL advancement taking place, but made mostly by developers who are:
Too busy creating complex software to release post-production videos BEFORE an actual product exists.
Working on mobile AR that can’t yet be released in the App Store, because of Apple’s restrictions (they don’t allow real time video access for image processing). In related news, NYARToolkit, the Java port of ARtoolkit, is now working on Android.
Looking five to ten years in the future and developing hardware that will help alleviate all the current limitations of mobile AR.
So, to sum things up, here is a list of suggestions (admittedly snarky) that I have for folks who want to write about Augmented Reality.
If you’re reviewing a mobile, location based AR app, actually take it for a test run to find a place and not just spin in a circle at your desk.
Don’t trust YouTube videos. Many of them are just After Effects jobs (i.e., AugmentedID). If the app isn’t released yet, try to get a live demo, or just add a disclaimer at the top of your article.
Ask hard (not really) questions like, “What service does this provide for the user?” and “Does this really need to be done in Augmented Reality?”
Don’t use the MacRumors model. It works for them, because at least some of their wild assumptions actually have the chance of being true.
Real advancements in AR will come once every few months or years, not days. Deal with it.
In all seriousness, there is long-term value in being a level head in a sea of hype. Please, be a voice of reason to your readers.
We Are Organized Chaos (WAOC) is Zugara’s (www.zugara.com) interactive marketing and advertising blog where we’ll be featuring some great projects and discussing upcoming trends in the digital world. Work — good and bad — will be critiqued. Hope you’ll enjoy reading our insights and thoughts on interactive.
