AR consumer research
Prof. Dr. Philipp Rauschnabel

I am Philipp Rauschnabel and have been a Professor of Digital Marketing and Media Innovation at UniBw Munich since 2018. Before that, I was at the University of Michigan Dearborn (USA) and at the Uni Bamberg, among others. My research revolves around Augmented Reality (AR) and related concepts (MR, XR, VR etc.). According to a recent ranking by Microsoft Academics, I am one of the top 5 AR researchers worldwide based on citations in the last five years. I am (co-)author of more than 20 scientific publications that provide conceptual or empirical insights into the impact of AR on customers and future management practices - and thus on society at large.

I’m a marketing field professor. Basically, when I get up in the morning one of the first questions I ask is what's happening in the AR Market or on the XR market today and how will it impact humans, and how humans impact the industry as a whole? I started with XR research almost 10 years ago. I was reading an article about Google Glass. It was one of these articles you read every day about anything that's happening in the world. I was asking myself what does it mean? Of course, I had a little bit of a naive understanding. I thought, like, okay! This is something crazy or something like Microsoft HoloLens nowadays but in a small device. And I was thinking that I’m looking at another Google device and everything around changes you. My expectations were of course way too high. But when I tried it for the first time the Google Glass device I was very disappointed. I began having this vision that there is virtual content around us all the time.
And when we look at the development of media… This is the next logical step, for example, to think of something like Facebook. Many years ago you used social networks on a desktop or a laptop computer and then you could use it on the mobile web browser or on one of the first smartphones. And then, suddenly, you are apt for that. You took a photo and then open it from your app and that changed… You took photos right out of your app. And then you added photos and text people. So, the boundaries between the real world, the virtual one, or the social media world disappeared step by step. The next logical thing is to get rid of 2D screens. The next logical step is to put content where we need it.
And that's not on a device like that. We need content where we are and where we try to solve certain problems with it and that's happening in the real world. I started working on these topics and at the beginning, I was more focused on social media. But the more I was thinking and understanding XR marketing - in particular, the AR Market - I realized that social media as we know it today is at the beginning of social media and of the way we communicate in the future. The future will be something that is more in real life. That's more how we like to behave and interact with things.
For example, if you are navigating something, if you browsing the website and you're using a mouse, this is so artificial! I mean you have a mouse next to you to control 2D content in front of you. Humans want to touch things and put them somewhere else. This is how we interact not with an abstract mouse! I think that this is just the next logical step that we make digital content more realistic, more intuitive, and more in the way how humans interact with content in everyday life.
Then I started doing research on that. Of course, a lot of people said – ‘why do you do that stuff? Focus more on traditional topics and don't do that futuristic stuff! I've never seen a person wearing Google Glasses in everyday life.’ People spoke of Google Glasses. They just continue doing that while we all know that's not correct. My mind is just exploding in the background! But I truly believe in that. Suddenly we see more and more researches and more and more companies get interested in that. Just recently we published an article. It is called “What is Augmented Reality: definition, complexity, and future”. In that article, we show a survey we surveyed managers. And of course, AR is not a standard in companies nowadays! But we asked those people who are not using AR in their marketing and those respondents said that they had some industry standards that represent the German field of innovations. But zero responses were of seeing no future in AR.
What we see here is managers have a vague understanding of what it is! They truly believe in it. But for some reason, they don't have the expertise, the time, or modern resources to really dive deeper into that. That's why I think that such podcasts like yours are extremely valuable to just spread the word about the potential and also risks that AR and VR can offer.

First of all, if I start providing that answer probably half of the people would disagree with me because if you asked three people for definitions then you will get at least four different definitions. What we realized a long time ago is that reality or virtuality continues and you can position everything between the real world and the virtual world. Everything in-between is called mixed reality. But there is augmented reality and augmented virtuality in-between. Of course, a lot of things have changed since that.
Then we have an XR term which is basically everything – AR, VR, or however you call it. And there are augmented virtualities which is a topic that's not very used quite often in everyday conversations. If you look at Google search requests, for example, it's basically close to zero. Sure, we all know what it means. But it's just certainly extended and disappeared from the everyday conversations. When we talk about mixed reality a lot of things have changed since the introduction of the Microsoft HoloLens device.
When Microsoft introduced the HoloLens device they tried to avoid augmented reality because people associate augmented AR devices and their variants with Google Glass. But they wanted to be different from that. They wanted to be perceived as something different. They used it to mix reality and explained it in a way that there is a mixed reality device is so great because it can do such a realistic integration of content in the real world. Basically what happened is people started associating the term Mixed Reality with a very powerful AR form. We're content that it’s not just really integrated. It looks realistically, there are multi-user AR, very intuitive interaction techniques. So what we mean here in theoretical terms it is a form of AR where content has a very high level of local presence.
Local presence means that the user perceives that the virtual content is actually there. And the opposite of that is something that we call assisted reality which is like text information. So you get a text with information asking to grab this product or get an email that's popping up or any other form of a note. It could be a text, a PDF, a video, or something like that. And that, of course, is also integrated into your perception of a real world but you don't believe that this content is actually there. You just know that there is artificial content there that assists you. We call this an assisted reality as a very simple form.
For VR it's basically the same! There are a lot of discussions. For example, is a 360-degree video a VR or not? Well, I don't think we can find an agreement on that within the industry. But in everyday language, we realized a lot of people would say that's a very simple form of VR content. And then we talked about VR. I think we should treat it as something separate and different.
In VR people are closed off from the real world. And, of course, not all VRs are the same. So again we see a kind of continuum arranging from very low levels of the art to very high levels. At a very low level, it could be something like 360-degree content. Of course, a lot of people could tell that 360 is not VR but there was a lot of discussion about that and whether 360-degree content is VR or not. We decided for us that it is a part of AR but at a very low level. Typically, it's 2D content that is smuggled in a three-dimensional way but now we’re dealing with people that associate it with VR. They say that it's a VR video on the Internet that says it is 360 degrees. And the opposite of a very simple form of VR is what we call a holistic reality, a holistic VR. That's something where people are in the world that's perceived as being actually there with a very high level of telepresence. While in the low-level VR which we call atomistic VR people always know that this is a fake world where they are in. And of course in holistic VR people catch VR content code with several senses. It's a separate Continuum!
Of course, from a technological perspective, you can ask developers why there are so many similarities between AR, and VR and how you develop the content. But on the other side, from a user perspective, being in an AR experience or in a VR experience is totally different. So do you feel it is different? I think when we talk from my discipline and from my research it makes much more sense to separate these two concepts - AR and VR - rather than presenting them on one Continuum.
And last but not least, the term XR is quite interesting. A lot of people say that XR means extended reality like a small anecdote but I wouldn't agree with that. If you translate the Extended Reality term to another language, you would probably get the same answer. Then if you translate Augmented Reality into another language then extended will basically mean augmented. But when we talk about extended reality, we also mean VR as part of that but in VR nothing is extended for it is replaced. So we would just call it XR but you will ask what X means. You could say that X is the variable X and you could place it wherever you want to – assistant, mixed, virtual, augmented, whatever. It's basically all new forms of new realities.

These are basically toothpick questions we try to answer in our research. Why do people with XR or AR and what they are doing with people? The second question is very challenging to answer because we can’t measure short-term effects like two people feel better or worse and how do they feel and how does it impact our privacy, for example. These are important questions we discussed. What does it do to the job markets and what new jobs will arise? What competencies do people need? All the marketing students know this and know how to edit a photo that is not enough. They need to understand how to model objects in 3D because they need to be able to understand what XR developers are doing.
And of course, we also look at how people get that and what people do with AR. How do they interact with that content? How do they shop in AR? What are the tasks? Do they try out more products before making a purchase decision or not, and do the anthropomorphized brand grows stronger? Today, human brains are more in AR! Does it perceive what sees as being closer? Do they experience products as touchable or not?
For example, we contacted one study with my colleague and looked at people who like touching objects in the real world. We expected that those people will hate AR because in AR everything looks real, you can touch these products but from the opposite. Do those people that like to touch products? So, these people that doesn’t like online shopping enjoyed AR significantly more.
These kinds of topics we address are very interesting. And the one is about finding the inspiration - Inspiration about getting new ideas and questioning existing knowledge. We passed across multiple studies part of them are already published. The other ones will hopefully be published soon. We could show that inspiration is a core construct of augmented reality research. Augmented reality is so powerful in inspiring people because they can change what they are and change what they perceive as realistic. We could show that if people get inspired they are substantially more prone to purchase a product. Most likely they will perceive a brand much better.
In one study that is not published yet we showed that people who are inspired in a one-time usage of the Microsoft HoloLens device showed substantially higher levels of refferity than exact same people in one week. One week to 10 days later we asked them of how this one-time experience impacted them. And they reported that were thinking about how this could change their life and how they will communicate in the future. We recognize that this inspiration construct is probably the most crucial variable in augmented reality. So it can inspire people. And that's very powerful. And if used correctly, of course, it is amazing for our society as a whole.

Actually, we conduct a lot of research on the dark side of my yard because we think we don't know these solutions and how to address these topics but we are currently trying to identify the negative consequences. One aspect, for example, is privacy. So when we look at privacy on our devices nowadays, at apps like Facebook… They collect a lot of data about us. They know who we are, what we do, where we click, and how long we stay on which website and they use it by doing so. They can estimate what preferences we have. Of course, the same goes for AR. But in AR we have sensors like here and even more censors in glasses and this end the track at the user's environment.
That's the core difference between the privacy issues in traditional digital media and AR. In AR we can threaten other people's privacy in addition to our own. And if you care a lot about your privacy, you could see that I'm not using Facebook. And then, of course, Facebook doesn't know too much about you. But if you say that you’re not using AR but someone else is using AR then you can't really escape from that. Especially not if we imagine a world where a lot of people wear VR glasses or contact lenses in public. That's one major threat.
We don't know how people will behave will people be more dishonest, for example, if they know they are tracked all the time. Or will they behave in a more ethical way because are observed all the time? We don't know how that impacts psychological stress. We conducted studies in work context where we asked workers with and without AR experience about the stress levels. And what we found is that as soon as people have AR experience then stress levels get reduced. AR is much more positive, which is a good thing because it's at least self-reported. But it's not always the truth. So what we're discussing is the negative consequences from an external perspective and how do they impact. What about their overall well-being or something people can touch?
Then everything about AR is more futuristic. Imagine we are using AR all the time and now are both looking on the monitor or the screen. Screen is the very unpractical device. Do we need screens if we have AR glasses? No! We could just have the screen, an app somewhere placed on a wall or somewhere floating around in the real world.
Probably some industries that are unrelated to AR might run into problems. Do we need a TV at home or could we have a Netflix app on the HoloLens device or on a smaller HoloLens device or something similar? We don't really know how this will impact existing Industries, but what we could show is that the idea of substituting real products is under one of the top priorities of many content and technology producers. We know that this threaten industries. I'm not really aware of that. We also know that consumers are surprisingly open to the idea of replacing physical products by holograms.
First of all, we thought - well, that's maybe a random finding in one of the studies and then we replicated it in different countries with different methodologies. We found very stable expectances right across countries and methods. This is another threat we have to be aware of. And of course there are many other topics that we just don't know right now.

We all know the rumors about Apple all the time. One week Apple is discontinuing the AR program but on the next day or the next week, you read that it is okay. In one month they will announce their Glasses. We all know these market rumors. I'm using a lot of social media like Twitter and mostly LinkedIn. I also have a couple of alerts that send me the newest Publications in Academia and I check blocks manually. In my classes, we have one exercise every week. Every week one or two students are summarizing the rumors in the market. Believe me or not they sometimes find topics and rumors that I wasn't aware of.
There's a lot of information in social media and Academia and the relevant tech magazines and blogs. I get a lot of information from people from XR Industries. They tell me they are working on this and this or take a look at something interesting… A new concept, they are developing.

Consumers like it. There are few studies that have shown for example that it could impact sales. There are a couple of studies that compared AR with traditional 3D or 2D presentations, and it was always superior. The problem is that when we talk about marketing nowadays we talk about product visualization like Ikea. Ikea is the standard example of how people can place a virtual coach in their living room. But I think that the real potential of AR is beyond product visualization for visualization is a tool to practice AR or to understand AR for consumers & companies. But the real value is not looking at products in 3D. The real value is something like we know from content marketing nowadays that needs good content. We haven't really understood what recipes are good for storytelling in AR but it needs something connected to your environment. We discussed in a recent article that the managers agree that it would be something around communities or the combination of AR content in social networks or another form people are using.

To name the most amazing one is very difficult. It depends on the purpose you have. Like the Google Translate app where you could translate another text is such a simple form of AR but it's extremely helpful if you're in another country. For example, while I was in Russia I could not read anything or translate anything but with AR I could do it. This is not a very sophisticated and very immersive form of AR. It is an assisted reality but it's extremely helpful.
On the other hand, I've seen an example of an education app of a WWII where people that survived the Holocaust were sitting in your house and explaining about the experience. It was extremely emotional experience to me. I learned a lot about that. But it wasn't funny at all or it wasn't cool at all. It was more like a sad, extremely emotional, it really captured my attention. Then I have seen another app where you could write virtual graffiti on walls not in a very realistic form. I mean how it was realized is not very great. But when you think of this idea, people can walk through the streets and comment on houses. And you think about the implications of that, the positive implications like, for example, restaurant reviews. So, rather than taking your phone and then searching for restaurant reviews you would just see the average star rating on the front door presented. These kinds of ideas are extremely fascinating to me.
On the other hand, when we think of the problems considering our being in WhatsApp groups nowadays… And if you translate this to AR then I think we will face almost all the problems and we don't really know how to solve these issues.

Many but not most are successful to a certain extent. We usually contact a lot of studies to test some theories where response either gets the AR version of a platform or not AR version. In most cases, the AR version performs better at least in terms of short-term effects. So, purchase intentions increase or willingness to pay or the attitude to watch the brand or how much people like the brand. We see that for almost all apps but not for all. There are a couple of reasons why this is the case.
First of all, some of the apps with AR content are not well-developed. The graphics quality is bad. In addition, sometimes we see something like a wall effect. I think that it is something that sounds very positive. But in fact, it's negative because the wall effect distracts from the content. When we started consumers using AR for the second time we gave them an iPad or something and they didn't focus on the content. They focused on what happens if I shake an iPad or if I assume to the avatar's ears. What do I see there? They try to play with AR rather than focus on the content. That is the dark side. The specific answer about the best AR marketing content I've seen so far I can’t really give because there are so many different ARs that are great on their own.

I do not think there a difference between AR and VR on a smartphone or tablet. I mean it's nice and it's very useful if you do self-augmentation like trying different looks before cameras and smartphones were created. For a long time usage of AR the smartphone is the wrong device. First of all, you need to hold the device and want to operate it. So basically you can't do anything as you can’t grab a product because you don't have a third hand. We need something as we call higher levels of a technological embodiment. So, basically, you forget if you're using VR technology. These maybe the Glasses or contact lenses. That's the first obstacle.
These devices are too expensive right now, and we don't really know how social norms will impact us. So we saw this as a good example with Google Glass many years ago. What people wear is just glassholes because they were wearing them on the street. And we know if other people don't like what you were doing. We should really change our behavior. That could be an issue but this could be solved by good marketing strategies. In VR we realized couple of different obstacles for the mass adoption. One is that people don't really understand what VR is. Sure, they put something on and ask what they can do with that. For some reason sometimes we have to say a lot of things but I can’t really tell you what. We lack of use-cases where people really see a value in that. We tried a lot of things like 360 degree videos but people don't like 360 degree videos at least not in movies because they are afraid they are missing important scenes or they want to use their phone while they're watching a movie or talk to each other.
Second, we realize that people don't like the idea of being isolated from the real world for a longer time. For a short time there is absolutely no problem at all. Maybe this could disappear once people get more used to that. And the third reason from my view is that VR sickness or motion sickness is still a severe issue. The research suggests that there are three factors that determine motion sickness in VR. One is the hardware. One is the software, and one is the person itself. We can solve hardware issues by higher FPS rates or a higher resolution and more computing power. We can also solve the content issue by providing better content that the people perceive.
The third problem is that some people are just more likely to get to experience some form of sickness while using VR. And these are typically the same people that experience sickness when reading a book in the car. We don't really know how we can deal with this issue and we don't have representative studies that show us the percentage of that. We also have people with problems with stereoscopic vision or some other problems with their eyes that could hinder the experience. But I think the signals issue is really a big problem and we realized that in our studies of how frequently you respond. Say - Okay. I have to stop. I have to stop at this point. I can't continue with that. I'm just feeling sick.

My first advice would be… No, let me start differently. If you had asked me the same question a year ago I would have said don't try anything, develop a strategy first. Having played around with AR while having studied AR for a while I talk to service managers. I would say what experiment with AR we’re going to do and then develop a strategy because the problem is a lot of managers don't understand AR. They associated it with Pokemon Go or Ikea place but they don't really understand the potential.
My recommendations… First one is that there are so many free tools to develop some very simple form of AR and we had very great experiences when having workshops with companies, managers or students and we just tell them to develop some sort of AR for we will never publish it somewhere but just do something. Then you understand what’s possible. Then we usually explain them what is the future. We showed them some examples and then would tell them – ‘Okay! Now develop an app that people would like to use and don't try to copy something that works in another media. Don't try to copy that on AR because we made the same mistakes in social media 10 or 15 years ago.’
We thought that advertising works in newspaper. We put advertisements and social media away for people don't like advertisements that didn't work. Then try to do something disruptive. Something that's totally new. And don't come from the top management. They come from new employees that don't know the standards of a company that don't know the existing processes that come with this audience. We also had good experiences and did have couple of creative ideas by telling people from a company to develop something that could kill your company. Try to find something that will disrupt your company! Then they got a lot of creative ideas. These were not final ideas but over a long period of time they turned into great ideas.

I thought of what the hell is going on! I was kind of in a bubble. I knew that everyone is talking about Meta and then I talked to my team to ask if people really know that. We had a survey about a different topic at that time. And I said that this includes a question if people know that Facebook change their name to Meta and most people didn't know it but those people have heard about it. They didn't really understand why they did it. They said it's like polishing their image.
I think the hype is very much in our bubble but not so much outside our bubble. What is the idea of leaving a 2D social network? Is this the right approach to do? Metaverse term is not very clearly defined. When you ask people how they define a Metaverse they could give you an example of a Metaverse. They will give you examples where we would say persistent VR in the past or as we call it a multiuser VR experience. A lot of people used to turn VR Metaverse for any form of VR app where different people can meet. To me, Metaverse is something that's more close to the country. You buy property, you have certain rules, and you have an exchange mechanism so you can purchase and sell something, create something, and do something with other people.
The topic is too new to say what it will exactly look like. I think we will have purely AR Metaverses vs. purely VR Metaverses vs. Metaverses that can be both. XR Metacerses, basically. I think that any forecasts on how the Metaverse will look would be not very precise.

Absolutely. I mean the question is will Facebook win the race or will it be another player. In the past we saw substantial changes in the media and technology landscape. We saw that new players or at least new brands entered the market. I wouldn't guarantee that Facebook will win the race but what they're doing is absolutely the right thing. They invest in a topic with a lot of potential and that's absolutely up-to-date. The future will not be a 2D Social Network anymore or a 2D image-sharing at platform or a 2D messaging chat. The future is in 3D. The question is - would it be more in AR or more in VR or equally in both. Will it be hosted by a commercial brand? Will it be hosted as a non-profit way of life? Will it be decentralized and not being controlled by an entity or will that lead to too much anarchy so that people will believe it? We don't know these questions but what they're doing is absolutely right.

I don’t think we can answer this question right now. I think we will see different solutions at the beginning like we did in social media when every company had their own discussion forum, own chat or own community, and then everything consolidated to two or three major platforms. If you ask me now, I would say that I expect something similar in terms of Metaverse and that we would have different solutions. Then they will consolidate to one or two in the future.

Good question! I don't have the perfect answer on that. The problem is that it is something you should probably more talk to a legal expert because the Metaverse is something that's not focused on one specific demographic & national area. National loss might be problematic to apply there. I think something like a community we see on Wikipedia could be the most effective tool. But if it's the best approach to protect a society then that's a different topic. How governments will react or the European Union? This is very unpredictable right now. I can't really give you a valid answer on that question. Maybe in a year from now our research will show something but so far we can't really tell you anything about that.

I don't think we can say this is the year of the Metaverse because we will have a diffusion curve and we need to figure out what we define as a Metaverse. Is it a Microverse or fully a Metaverse? Or is it something that is bigger? I think it will start within the next few years and then it will make its way. I don't think it would be scientific or accurate to say that this is the year of start or a next few years. It depends on the progress on the hardware side and on the software side. But at least we see that this is not representative of what I'm saying now. But when I talk about these concepts with my students - and I do that since about ten years - it was totally unknown few years ago. And just today I had a long conversation with my students who invest in NFTs and several platforms that we could call some sort of a Metaverse. We see that the innovative consumer groups are taking the Metaverse concept seriously nowadays and that's the first attempt to make it ready for the masses.

Questions I couldn’t answer are basically about the future. How will the future look like? And the only thing I can guarantee is that the boundaries between the real world and the virtual world. They will disappear the same as the boundaries between digital & analog. They will merge. This is what I can promise. But how this will exactly look like is totally unpredictable.