Next-gen immersive technologies are the future, but a lot of people don’t really believe in them, thinking of them as niche applications and expensive parlor tricks only. But in truth, they are the culmination of a trend that has been over 100 years in the making. There are hurdles, as there are in many races, but they’ll be overcome. Immersive tech will happen.
Immersion is not new
A medium in its most basic sense is “a means of communication or expression.” But humans leverage media for two purposes. One is communication, the other is immersion. And while immersion is a cutting-edge concept in the tech world, it is very much an elementary concept in terms of human history, as immersion is something storytellers have been trying to achieve since we acquired the power of speech. Any good storyteller knows that a simple relaying of events, either real or imaginary, is not enough to earn your supper. It is immersion, the transportation to another world through the relay of its various sights, sounds and feelings, that separates a great story from a good one. The same applies to other forms of entertainment that leverage media, such as music or the printed word. Their most effective examples immerse us, either by taking us to other places, or bringing other places to us.
Modern media and the tech that powers it started ubiquitizing in the late 19th century, and ruled the roost for nearly 100 years, strong & unchallenged. Whether print, radio, movie, or other, all were characterized by their “broadcast” nature. The sender sent and the receiver received, largely in a one-way fashion. Television was probably the medium most vilified, but all were essentially the same, with the receiver told what to think & feel with little-to-no power to respond.
But then in 1970’s and 80’s something happened – disruption, arriving in the form of the video game. Suddenly it became possible to control what happened on the screen. Some might say that those games don’t qualify as media since the video game units of the time were closed systems, but disruption was happening regardless. A new form of screen entertainment was emerging, far more immersive than what came before. Want proof? Gaming has a much weaker correlation with obesity than TV viewing. There’s plenty of time to walk to the fridge when watching TV. With gaming, there may be none, and many are too immersed to stop, even if given a choice, even if hours have passed. It’s a whole other level of immersion, with time vanishing in the same way it does when artists compose or old friends talk.
Personal computers started arriving soon after, with the Internet arriving soon after that, and they have proven more than gaming’s equal as a time suck. And now we have a whole generation (digital natives) who have always had these interactive technologies at their disposal. And for them, two-way interaction isn’t just a means of entertainment – it’s a way of life, with immersion hard to achieve without it.
Immersion has become a tool
Historically, immersive tech has largely been used to entertain, with its value determined likewise – you’ll switch on your surround sound system to watch your favorite Hollywood blockbusters, but would you bother with it while watching the news? There have been exceptions of course, with the prototypical example being the flight simulator, but even these have been quite rudimentary until fairly recently when it comes to immersion. Largely they’re used to hone pilot reflexes, with occupants rarely forgetting they’re in a simulator.
Gaming and computers no doubt laid the groundwork for immersion as a tool. They brought the user much closer to the screen than was normal before (filling your vision in the way a movie screen might) and provided a level of interactivity that was compelling. It didn’t even always have to be fun to immerse (those of you who remember the many hours lost spent spelunking in the caverns of DOS circa-1994 can attest to that). At whatever stage you first started to experience it, the end effect was the same – the virtual world on that screen took on the same import as the real world that surrounded it (at least while the screen was on). Once that happens, immersion need not entertain to be achieved. It just need be available.
Hurdles to ubiquity
Next-gen immersive technologies may come across as radically different than what we have now, but according to tech blogger Tom Goodwin, they actually represent the logical conclusion of a trend that’s been going on with screens for over 100 years. Screens keep getting closer, from cinema screen to TV to smartphone to headset, with the degree of control and interactivity increasing and becoming more human each step of the way. In other words, technologies like augmented reality (AR) and virtual reality (VR), where the screens usually sit next to your eyes, are the future. However, just because something is inevitable doesn’t mean it will be easy. Five hurdles have to be overcome for immersive tech to ubiquitize.
Professor Byron Reeves at Stanford University recently ran a study that sampled the laptop & smartphone habits of the student body, and found that the median time spent on any one task to be only nineteen seconds. This is only the latest in a long list of studies that confirm this growing trend towards “switchiness” amongst Homo sapiens. So what does it mean? According to Tom Goodwin, “We’ve become addicted to stimulus; we’re now so overmediated that our attention has never been less focused. Partial attention isn’t just possible, it’s our default position. How else can we now consume 15.5 hours of media a day and manage to sleep and eat? Our attention is shifting to snatched moments, the quick glance.”
Why are we living this way? According to Professor Reeves, task switching can actually be interpreted as an act of autonomy over our lives. “People like autonomy. They are in control. They can decide how they move from one type of content to the next. They don’t have to wait for your message to end, they can end it, or hit the pause button in between.”
Reeves would later add, “I’m very interested in whether immersive experiences can win over the tendency to switching. You could imagine that I have a set of VR goggles, I’ll be so engaged that I will forget to switch. And I think that is a favorite thought that everyone developing for that technology has. But, you might keep the goggles a little bit longer, but I think the interest in coming and going will still be there.”
Screens are proliferating at an incredible rate, and this no doubt is contributing to all that aforementioned switching. According to Time Inc., digital natives switch media platforms (many of which involve screens) once every two minutes, while those age 35-to-55 switch once every three minutes. The problem today is that each screen is typically connected to a different ecosystem, operating independently of the others. This means that a lot of potential synergy being left on the table. What does this have to do with immersion? How immersive will a VR experience prove if you have to keep taking your headset off to answer the phone? How immersive will a 4K program be if you have to keep looking at your tablet to see its Twitter feed? Other examples abound.
Each next-gen immersive technology, whether 4K, AR, VR, etc., has a raft of user-end technical issues to overcome, but those are discussed in this magazine’s individual articles devoted to them, and need not be addressed here. Here we’ll talk more about the networks that will bind all these technologies, and the issues here are two – bandwidth and latency.
Immersive technology will require massive improvements on both fronts. Today, Huawei has a remote-reality prototype (MirrorSys) that requires 100-to-200Mbps for a one-way immersive experience, and that’s just the tip of the iceberg. MirrorSys uses a single 8K wallscreen. Michael Abrash, Chief Scientist at Oculus VR, estimates that photorealistic VR will require two 16K x 16K screens (one to each eye). You do the math.
Latency is the other big issue on the network. With an augmented reality headset, for example, real-life visual and auditory information has to be taken in through the camera and sent to the cloud for processing, with digital information sent back to be precisely overlaid onto the real-world environment, and all this has to happen in less time than it takes for humans to start noticing lag (less than 100ms). Today, network latency could be up to 20-times that. There’s much work to do.
So why do networks need to support this, anyway? Today, immersion is largely a solo affair, carried out in closed systems. But digital natives (young people today but soon to be the majority) will demand something more. According to Tom Goodwin, “The notion of the mobile as a second screen was common in the early 2000’s, now we see mobiles and wearables as the primary screen, with the other screens fading into the background.” In other words, our primary screens are now the connected ones, making connection the default expectation from this point on. Why? Goodwin would add, “Something about proximity and our demands for the immediate make our closest screen the primary access point.”
In a headset scenario, the closest screen is next to your eyes, so it had best be connected, interactive, and as amenable to personalization as possible. Huawei refers to this as ROADS capability (real-time, on-demand, all-online, DIY, and social). Digital natives demand ROADS capability in everything that they do. Immersive experiences will be no different. Those who ignore this do so at their own peril. As stated before by Byron Reeves, our newfound affinity for switching tasks will never be completely overcome by immersive technology, as people will still “want the autonomy to come and go.” He sees interactivity in the experience as a must. “The best gift you could give me in a conversation is something that indicates you understood me. So, we are truly interacting. We say there is contingency. So more interaction, more contingency in the interaction, absolutely creates more reality.”
The path to immersion has included a few potholes. VR has had a few false-starts thanks to unrealistic expectations, poor experiences, and occasional nausea. AR took a hit when the word “Glasshole” entered the lexicon. Plenty of experts today see immersive technologies as solutions in search of a problem, or niche applications that won’t ubiquitize in the way that smartphones have.
The way ahead
But remember, these five factors are not blocks, just hurdles, and they’ll either be knocked down or surmounted in due time.
An integrated cloud framework
Both the “switchiness” and “screen chaos” problems have basically the same answer. According to Huawei Media Lab Head of Strategy José Alvarez, “An immersive experience is an integrated experience. And for an integrated experience, you need a framework in which you can take all of the info that you see that is useful to you today and bring it to a single place. Today that integration doesn’t happen because there’s no common platform. When we talk about really immersive tech, we have to think about all of these experiences in one place. Your phone rings in your field of view; your data and friends in the same field of view. If you are watching a movie, or playing a game, and you have a phone call, the game (or movie) is automatically paused. You don’t even have to think about pausing the movie and answering the phone. You can make those decisions, but you can only make those decisions if they are integrated in a platform that provides that kind of information. The role of telcos would be a single place where all customers can come to get all of their services.”
Huawei Media Lab Head James “Bo” Begole would add, “The telcos can provide the kind of network infrastructure that enables a fully distributed cloud immersive experience where a lot of the computation can happen on very powerful servers that are in the cloud while sharing the sensor data that are being delivered by end-user devices at the client side. If we can get devices to share enough information so that they know when and how to interrupt the user at the appropriate times, then users can find this space to spend all of their attention in one immersive task. So they can be more productive or more entertained.”
This may sound like a tall order, but remember that technology trends such as cloud technology and the Internet of Things (IoT) are already driving a greater need for openness in terms of interoperability and data sharing. Immersive tech will be no different.
5G is coming
Fixed network capacity for immersion shouldn’t cause any sleepless nights. The theoretical data capacity for optical fiber is almost infinite, with 1Gbps to the home already available today in markets like Hong Kong, and G.fast technology soon able to match that via copper. But the modern lifestyle is mobile, of course, and immersive applications like AR will very much depend on it. Huawei is working to bring 5G, the next-generation of mobile, to life by 2020, with the goals of 10Gbps data transmission and 1ms latency. These networks are being designed with self-driving cars in mind, and immersive tech can have similar demands in terms of speed and response times. In fact, a self-driving car could be the closest thing many of us will have to an ideal environment for a little VR immersive fun. ROADS-capability at all times within 5G coverage will also be possible, but it will be up to the application designers and interface providers to make it happen.
Of course, killer apps are a must if immersive tech is to avoid being sidetracked into niche markets, like the naysayers predict. The nature of the app will depend on the medium. With next-gen video, it’s hard to say. Next-gen video will be an interactive experience, a tool; some people love tools, other prefer to watch others do the work. But vertical applications are sure to be huge, as the benefits to surveillance, remote operations, telemedicine, etc., will prove too much to ignore.
For AR, it probably won’t be an application, per se, that proves killer but a scenario – new situations, especially those where we tend to be timid and consult guidebooks or smartphones. We will be able to stride more confidently, into any new situation, because we will have a lot of relevant information at our disposal; we won’t feel so lost. According to José Alvarez, “With augmented reality, we will feel empowered. We will feel that our cognitive capabilities have been improved. No longer will we go to a new place not knowing where to turn. No longer will we go to a party thinking, ‘I’ve seen this person before. I wonder what his name is.’ No longer will we need to worry about those things. And those things will be empowering, by freeing us to do more things with our lives.”
And for VR, the killer app will be nothing less than a virtual Internet itself, known in today’s vernacular as “the Metaverse.” Both Facebook and Google are hard at work positioning themselves to be the potential homepage for it. Virtual Facebook would mean a far more social & interactive form of social media. Virtual Google, for starters, would mean more fluid and intuitive forms of search. What would they get out of this arrangement? Many more billions of dollars, for one, as a VR environment would enable precise tracking of whatever the user is looking for every second the headset is active, as opposed to tracking of the occasional like & click that we have today. This would be a gold mine to advertisers, and those who profit from advertisers. According to Tom Goodwin, VR is “the most personal experience with the closest screen, providing the most connected, most immersive experience the world has ever known and can ever imagine – a chance to finally capture attention, focus the eyeballs, and build relationships with the most captive audience one can hope for in this age.”
And don’t forget, people like immersion
Immersive tech has had some growing pains, but what’s available today is far more slick & seamless compared to what came before. VR@Berkeley is an undergrad student group at the University of California at Berkeley devoted to the development and proliferation of both AR and VR. Why do this? In part, because “we live in a physical world with physical people, but increasingly our interactions are mediated through digital means: we find ourselves and others looking down at our apps, at our phones, at our laptops, and so on. Virtual reality challenges that relationship with technology. Instead of interrupting our lives to interact with technology, technology should better understand and accommodate us.”
Are people receptive to it? According to Club President Daniel Pok, “Everyone that we’ve shown it to has received it positively. VR promises mixing of the virtual and physical worlds in a way that doesn’t have to interrupt our physical lives.”
When asked to describe how people react initially to VR, Club Research Team Member Isabel Zhang stated, “From my own experience, I was just amazed as to how believable it was. You get really excited about how much further we can go into immersion. I was demoing Google Cardboard (a relatively crude form of smartphone-enabled VR) and little kids would came up and they would not put it down and let other people use it because they were so excited that they ran around showing everybody they could. It was really exciting to see other people that excited and happy about something that we’re so passionate about.”
Not convinced yet that immersive tech will ubiquitize? According to Jez Jowett, Global Head of Creative Technologies at Havas Media, VR is expected to reach 500 million users by 2018, scaling six times faster than the iPhone. Skeptics call VR the same thing they called gaming 30 years ago – an amazing toy for boys. Today, more time is being spent gaming on smartphones than doing anything else, and women are gaming more than men. That’s ubiquity.