SAINT ELMO's
SAINT ELMO's

News

Frictionless AR Experiences

A new world is being created

“I regard it as a big idea like the smartphone … I think AR is that big, it’s huge. I get excited because of the things that could be done that could improve a lot of lives. And be entertaining. […] Imagine if the line between the virtual and the real simply did not exist. Your classroom could become the cosmos. The past could be as vivid as the present. […] And this is just the beginning. Welcome to a new world."

These are the words used by Apple's CEO, Tim Cook, to describe AR (Augmented Reality) in an interview with The Independent. The man isn't skimping on the superlatives; obviously, he's completely convinced of the technology and the possibilities that it offers. Will Apple use AR in the future? Apple is under pressure to deliver the next big thing. The sales figures of the iPhone, the cash cow of the American company, have recently dramatically collapsed. The tech giant's share price is falling – and tech gurus around the world are asking if Apple's star, more than 7 years after Steve Jobs' death, has finally passed its zenith.

One thing is certain. Fewer and fewer differentiating features of smartphone generations are attracting fewer and fewer users to follow the trained biennial rhythm and buy (ever) more expensive iPhones. The Apple myth is no longer sufficient for purchasing motivation.

The next iPhone will not be a phone

But what if the next iPhone isn't a smartphone at all? What if Tim Cook proves the opposite to his doubters? Are his strategic investments possibly an indication of AR as a key technology for a completely new product category?

Rumors have it that Apple is planning its own glasses for AR and MR (Mixed Reality). Data glasses that follow Apple's aspiration to aesthetically combine technology and design while maximizing user-friendliness would have the potential to prepare augmented reality for the mass market. And could not only complement the iPhone, but even subsequently replace it.

In any case, with the ARKit, Apple has integrated the technological basis into the iOS operating system. This means that all iPhones from the past two years can use AR features. Google is using ARCore for Android smartphones a similar way. Thus, by 2021, almost 3 billion users will be able to use Augmented Reality preinstalled on their smartphones and tablets.

Considering that the computing power of current smartphones has risen exponentially in recent years, data glasses such as Hololens, Magic Leap, HTC Vive, Oculus Quest (coming 2019) and even an Oculus Go may soon be more imaginable for daily use than traditional devices.

Computing tasks such as the rendering of 3D visualizations could in the future take place on the smartphones in our pockets and the data would then be sent by wirelessly to the glasses – on which "only" the projection into the field of vision of the wearer and the sensory detection of the environment will take place.

If Apple manages that, then bravo! This would easily make all wired devices obsolete. That would be a truly groundbreaking product that would have the power to replace the iPhone as a disruptor.

But why should the future lie with AR and not VR (Virtual Reality), the technology that's now on everyone's lips?

Thousands of flies on a pile of shit can't be wrong

Crucial to the success of a technology is its penetration. How many potential users have the opportunity to use it and how simple is it? The less complicated set-up or additional equipment that's needed, the more likely a technology will be adopted by the masses.

Eleven years of iPhone (and subsequently smartphones in general) have all spoiled us so much that our expectations are extremely high. An app that does not open in 15 seconds, load times that are more than 3 seconds, all of which immediately leads to frustration and rejection. Fast and easy or forget it! – The today's motto.

VR is cool, but it requires a special headset or VR glasses. And if necessary, even sensors that need to be placed in the room. It's expensive and cannot be used anywhere and at any time. Many VR headsets are also useless without a connected computer. The use of VR is already limited per se.

One of my mentors once coined the most wonderful marketing slogan of all time, "Thousands of flies on a pile of shit can't be wrong." That is, people go to where something is already happening, where the way is easiest for them. An in-house option for integrating AR on a smartphone, which is integrated into the operating system, is an argument for the technology that should not be underestimated. (Consider: By 2021, 3 billion users are predicted with AR-enabled smartphones.)

Users get used to a technology, but also by being confronted with it again and again. Apps in the areas of entertainment and gaming are already creating mass-suitable points of contact. Pokémon Go opened the door to augmented reality for everyone back in 2016. In 2019, the unofficial successor "Harry Potter: Wizards Unite" will be released with more realistic AR functionalities.

Gradually, generations of smartphone users will increasingly come into contact with AR until it comes naturally as a form of interaction with their surroundings. The entry into more complex AR experiences or AR experiences integrated into daily life is then just a stone's throw. Unfortunately, VR didn't have the powerful clout to penetrate the market.

Mobility is king

In the pantheon of technologies, concepts, devices and application scenarios that currently coexist, many things may be confused. VR, AR, MR – or the common umbrella term XR ("Extended Reality"): Each technology has its advantages and disadvantages and is better or less suitable for different target groups, topics and applications. In addition, the boundaries between the technologies are often fluid. For companies that want to use them in projects, independent consultation is often advisable.

Complex and immersive training applications are rightly taking place in VR, where users – in addition to a stable stomach – have enough time for a briefing and the experience itself. However, the time of big VR optimism in the entertainment sector seems over, if we're to believe IMAX Corporation, which wants to close a large portion of their VR event and experience locations that debuted in 2017 in the first quarter of 2019. It's understandable. Because why should visitors pay a premium price for an experience that's only marginally different from a home gaming experience?

Mixed reality is the most impressive technology because it recognizes real space and mixes it with virtual objects in a way never seen before. The MR headsets of the current generation are still too expensive for broader market penetration. In addition, the technology still has significant development steps to be done.

Looking at AR and MR, superficially the technologies are very similar. Going into the details, there are differences in the degree of immersion and possible applications. The (currently) significant advantage of AR is that it's applicable practically anywhere. Because on a smartphone I may only have a comparatively small screen, but I can always pull it out of my pocket. Fast, uncomplicated, and without extensive preparation – just the way users like it. Open the app, hold up the camera and I'm already experiencing AR content. For 99 percent of the users, that's enough and it's unbeatable in terms of mobility. This fact also makes "mobile AR" the driving force for an entire industry.

Of course, there are also specific applications and "larger" AR scenarios (for example, projected AR, AR on built-in screens, etc.), which overcome certain limitations for AR on the smartphone. Because, let's face it, who's going to hold a cell phone in position for more than 10 minutes to experience a scene with augmented content? Because at some point, your arm is going to get tired. That's why short augmented sequences make the most sense.

But there's one more obstacle to overcome. Users usually need to install an app to experience AR on their device. Such apps exist today for different purposes. From contextual information presentation, for example in navigation apps, to the additive visual changes for selfies in real time. SnapChat has gotten big with it and has popularized AR experiences where users can show their friends and the world what they look like with rabbit ears and big cute anime eyes. But that's not the end of these developments.

AR without apps on the Internet will tip the scales

What would be the perfect AR solution for the users? – As a user, I would like to start AR experiences without much effort. Since I'm already on the move anyway, I would like to get the best contents from everywhere and ideally without lots of hurdles or at great expense. "Point and shoot", so to speak. I can offer this one web-based AR, that is, AR experiences that work in the mobile browser.

Advertisers may also be interested in providing users with content, stories, and experiences wherever they are. And over 70 percent of all page views are done on the go.

Still, it takes a lot of effort to get a user excited about an ad for their own AR theme, then get them to install an app – even just asking them again to scan the ad to enjoy the AR experience. These extra steps usually result in high termination rates.

With HTML, Javascript and a little bit of WebGL magic, it's possible to experience augmented reality without installing an app on your smartphone. Using the camera, holograms anchored in the real world and 3D objects as well as face masks as we know them from SnapChat. Because even a face scan is possible – everything as with AR experiences, which were previously known only in native apps.

Currently, a standard is emerging that will be adopted as WebXR 1.0 by the W3C and will enrich the previous approaches with additional functions. Then AR, like videos, will become the common format on the Internet and all vendors can agree on a consistent implementation of mobile WebAR with their browsers. Users and advertisers will be pleased.

"Frictionless Experiences"

Web-based AR makes it significantly easier for users to access their AR experience. For example, they simply come by QR code (finally this gets permission and a second chance) on a WebAR microsite and they are already in the AR experience. It simple can't get any smoother or more direct, and as such "more frictionless".

That's practically a revolution. In a variety of areas, a great deal more is becoming possible, things that were previously considered unimaginable. Whether seamless media changes from print to AR or from large screens to AR, whether in industry, retail, training, museum communication or service communication – wherever a fast conversion to AR is required, solutions are now in sight.

Apple also holds the next trump card in their hands: the purchase of Shazam in 2018 could one day make it possible to use images as codes and markers to make WebAR experiences even easier.

Conclusion: Everything indicates that AR will sooner or later become established in the mass market. Advertisers can already be prepared for the fact that accordingly the user demand for new AR experiences will increase. And with WebAR come new formats on the Internet, which could kick the video as "king of content" from its throne.

Share article

Talk to us!

Kevin Prösel Managing Director

Ziegelstr. 16
10117 Berlin
T: +49 (0)30 6953570-30
F: +49 (0)30 6953570-99
Email: k.proesel@saint-elmos.com