How XR will transform healthcare with Ph.D. Walter Greenleaf

"I am a neuroscientist and digital medicine expert.
My passion is to explore and develop new ways to apply technology to address difficult problems in healthcare.

My academic positions:
Distinguished Visiting Scholar - Stanford University's MediaX Program
Visiting Scholar - Stanford University Virtual Human Interaction Lab
Director of Technology Strategy - The National Mental Health Innovation Center

In addition to my academic work, I strive to move VR/AR technology forward by serving as a technology and neuroscience advisor to medical product companies, investment groups, and accelerator/incubator programs."

I'm good. It's a pleasure for me to be with you. Thank you for inviting me in. Today I'm in an especially good mood. We’ve been working hard for the last few months together on a book that gives an overview of medical applications as well as of AR, VR, XR technologies. Today I’ve just got the pre-publication version. It’s pretty good. The title is «Applied Virtual Reality in Healthcare: case, studies and perspectives». I'm the editor but there are a number of fantastic pictures put together in chapters for this book.

There are a number of initiatives at Stanford looking at applying XR technology to Healthcare. For Lab, I'm most tightly centered on the virtual Human Interaction Lab. In that lab, we look at how VR, AR, and XR technologies can be used to both influence our behavior & attitudes and also how we can study human behavior & attitudes using VR, AR, and XR technologies. It was founded by Dr. Jeremy Beylinson who works in this field fighting for about 15 years or longer. Amazing studies looking at how well immersive technologies can make a difference in people's attitudes, shift attitudes. A lot of his focus and his research associates’ is on how can we use it to tender traps around biases, how people can learn to shift their behavior and make it shift (like maybe being more conscious of that). You know… It’s how we treat our friends and people we meet. So it is in the field of Behavioral psychology.
I also do some work with Stanford in psychiatry with Kim Bullock. We have a program where we're looking at how VR & AR technology can be used clinically to help people who have problems in psychiatry & psychology. We help with anxiety, help with depression, and help with some other areas in that zone. I also do a little bit of work over at the Stanford Children's Hospital looking how VR help in a variety of ways: distraction from pain and teaching how to manage anxieties or how to manage chronic pain. So, that's what I do at Stanford. But I spend most of my time outside of the University.
My work right now is largely taking work within the university environment and not just in Stanford but that many other universities worldwide and helping to connect that with pharmaceutical and medical devices, hospital networks to help translate what we found in the research labs and get it out to use. And that's kind of really wonderful thing for me to see because finally that's happening.

I guess my answer would be yes. It's always hard with scientist to get a consensus for their job is to be critical and to think of next stage of things. But I would say that the research community, clinical community, hospital administrations and people who are writing the checks for healthcare are all starting to see the value of VR and AR technologies. There're many people who have made it a major point of how they do provide Healthcare. So, yes both on the science level and on the clinical care level. I would say we're at the early adopters’ stage because the technology is not very powerful and useful but people are getting more comfortable with it and there are still some issues in terms of building out more content, more applications that can go more. And there're also issues of that everything are on a different headset. So if you're trying to set up the use of VR in your clinic then you don't want to have one headset that does one thing but to run another application you have to own a completely different headset. So we're still looking for some ways to break through that barrier. But in general, yes! I'd say we're well over the threshold of where the science shows the value of these applications for clinical care.

I’d say that sectors are changing. I think that Healthcare has a tremendous amount of inertia. And in order to change how something is done in healthcare you have to not just show through research its value for improving Healthcare but also that it doesn't cost too much. It doesn't take too much time for clinicians are very busy. And that it's safe and effective also. And so we had the VR & AR technologies really in the research zone for decades. So we we've been able to do the basic research there but we still need to do more of that. I'd say that business and economics are showing that the value is in giving people a chance to understand how it can be incorporated into their practices but there are some great examples of where it's making big difference. It's not just the VR, AR, XR technologies. It's in combination with wearable sensors with machine learning. It's a number of 5G and a number of other technologies that are emerging in confluence. They allow us to do these powerful things. So I think we are at a tipping point. There are some leading hospitals and clinics that have some amazing things with it. But it's going to take a little while before it gets out beyond these early adopters.

No! You're right. That is a big problem and one of the things we need to be able to do is to look at the patients in different locations in different context and be able to compare them with norm of the data or with their past measurements. And we just don't have that in place. It's a little bit like when the medical imaging systems emerge with x-rays, scanners, FMRI machines. If they didn't have some standard ways of sharing data then there would be a problem. So it's a bit of a challenge there.

I think right now there is a shift in how people interact with clinical community in many ways. One is the center of care now is shifting from the clinic where the doctors are. Now, of course, for some things like neurosurgery you have to go to the hospital and have to be with their specialized equipment. But we’ve learned now that a lot can be done in a telemedicine manner with wearable sensors with immersive environments that can create an experience for both clinical evaluation and also clinical therapy. We think that the patient become where the care occurs. And now with the technology companies moving into Healthcare coming up with wearable watches that help diagnose or help people manage their own health and wellness. We’re again seeing that the person become more of an agent to compare their manager Healthcare and the technology is following them to allow more things to be done remotely in a telemedicine manor.
The power of immersive environments is that we really can cognitively engage and also physically measure what someone is doing. So it becomes a fantastic platform for distributed Healthcare. It's some kind of a shift for the clinicians to get used to that change and for the patients to get used to that change and for the whole ecosystem to shift. So, even though it's happening it's not going to happen overnight.

It's a difficult question because there're so many amazing areas I tempt to say in terms of surgical planning because surgeons can do a better job of planning a complex operation and that's important. But it's also making a big difference in terms of healthy people who are in pain to evolve pain and also a chronic pain management. I'm excited about what's going on with treating post-traumatic stress and depression. Those are very difficult problems to treat. Otherwise, addictions… We have some great ways of helping people learn situational confidence and refusal skills. If we can evoke the cravings for someone we can teach the skills to manage that. So, I could go on. It's a hard question to answer. What's the most important? I guess it depends on what you are most worried about. But in almost all areas of critical-care we're seeing some big improvements. I will maybe make a categorical conclusion as to what I think is most exciting… That's the new ways to do assessments, to do measurements. Immersive technologies allow us anything instead of asking someone how they're feeling or after using a paper and pencil test to come up with your evaluation of their cognitive processes or, you know, observing how they move from with AR, VR, XR technologies. We can challenge them, see how they react and make it a more objective measurement of how the brain is working and how their body is working. I think all that is important to allow us to come up with a new generation of therapies, and a new generation of pharmaceuticals, too. Because now we can do our researches on what medications work better with more accurate data.

Okay. In terms of where the data shows that we can make a big difference and what's been studied… The longest has been the way virtual environments can help with psychological issues such as helping with phobias, fear of heights, fear of flying, fear of spiders. And also we’ve had a lot of research showing this value of getting pain even including seeing what changes in the brain when you use immersive environments compared to standard therapies. I'd say those areas have had probably 15 years’ worth of research publications. In terms of where the research is, I think that's the strongest center gravity. But all the other areas that we talked about also had substantial work showing their value.

Well, I think what I've seen is… One project I was pretty excited about is at the Stanford Children's hospital. We adopted system to help adolescence and younger children who were scheduled to have heart procedure weeks before for them and also their family members. What we did is we captured, and this was not very difficult thing to do, 360-video starting in the parking lot of the hospital going into the check-in room, the waiting room into the surgical preparation room, into the operating room and then the recovery room. And we made other guides to our children of the same age and gender that show the patience to show them what was going to happen and then to talk to them about it. This ability to see what was going to happen and be mentally prepared for it reduced the anxiety in advance not just for the patients themselves but also for their family members. And it also allowed some of the children who were refusing saying, you know, this is something that I just won’t do. I know it's better for my health but I'm too scared to do it. I'm not sure what's going to happen. Allow them to get over that fear. So, in terms of helping something specifically just using a very simple thing of having people get a tour to clinic, see the equipment and that it was so easy to do and it made such a big difference in terms of the health care for these children.

Well, I think it's as someone who loved reading «Snow crash» and some of the other books describing the Metaverse and its concept are of course excited. In my opinion Healthcare is a major way we leverage immersion of technologies. But really, I think our whole world is going to be transformed by these technologies in a positive way. So, having large technological companies like Facebook, I think makes sense and is very appropriate. I think that it's just going to be on the way to the future and I hope that were now the head of it a little enough. You could design to make sure we don't build biases, that we don't have problems and that we pay proper attention to security and privacy especially in healthcare arena. I think right now we can probably figure out personal identities displaced on voice tone. Movements within a virtual environment might allow us to identify who is who and we need to find ways to both saw a lot of people to work for problems with addiction or alcohol or be part of, if they're in a dictatorship and they are a dissident, to make sure that their identities aren't revealed just because they're part of that Metaverse. So, yes, I'm so excited about the potential and I know that other technology companies will be moving into this area, too. But I also hope we can get it right.
And what I have seen in the healthcare arena is concern. Right now Oculus Quest 2 is the best headset in terms of specification and price point. But many of the clinicians that I talked to are concerned they don't want to have to ask a patient to have a Facebook account in order to use the therapeutic device. Also, educators who are teaching students don't want to have to say - to be my student you need a Facebook account. So, there're issues about this, but overall I think this is how we're going, and it's exciting to see that technology companies are helping.

I agree completely. I think we have to work hard to make sure that we are doing this right way. Technology that we're setting now will be with us for decades and for future generations. And we have to be wise, clever and fair about that. We have to also realize that that transcends a nation, that transcends ethnic groups, that transcends age groups. If we need to make sure that we don't have barriers we won’t exclude people from the part of this very important thing. So there's a lot of work to be done but I think in general it's good to see that there is some commitment of the technology companies to help us move there. And what we can do is help this do right.

Absolutely! I think there are still some issues right now when you put a screen close to the eye, you're fooling the brain in terms of the dimensionality and there's a disconnect between the eye and vestibular system. We need to make sure that especially younger brains are having this experience and are developing and changing how they see the world that were not creating something that could be unhealthy for these emerging and developing systems in our youngsters. So, I think there is a lot more work to be done on the safety of spending a lot of time in a virtual environment using today's technology. Now the basic principle of having shared environments where we can meet each other, we can share information with each other and do amazing things. I think that basic principle is fantastic. But I just want to make sure, and I know that you do too, that it's safe on all levels.

It's always hard to keep up with what can happen in a Steven Spielberg movie. I think that our expectations will be met. I think we will have an amazing virtual need to socialize, to explore, to work, to be entertained, to grow, to a health care delivered. I think it will be. I don't think it will be dystopian. We will be more creative as content creators and we can make a wonderful experience for people. I don't think it's going to be how people choose to spend all their time. I think that is wonderful to meet people in that way. The real world is hard to compete with that.

In many ways it reacts very positively. Several large pharmaceutical companies with significant investments are looking at immersive environments for healthcare and not just once that are used in competition with her medications. But just because I know that they help how to distribute and how to validate the safety and efficacy of an intervention. They're saying we know how to build great things in healthcare and let's invest in this. We've also seen some medical device companies make the same decision saying we know how to upgrade products, let's go into digital health and into VR, AR, XR technologies. So, the reaction from some of the large Healthcare Network is the same thing. They said that we're not just going to wait on the sidelines. We're going to go to best and design and get involved and try to make sure it's done the right way. Not everyone but few pioneering pharmaceutical medical devices. Healthcare networks & clinicians & research groups are saying - let's do it and let's do it right. So, I think I'm optimistic. I think that it's a new technology that can make a big difference but it does also can be designed to be utterly compatible with existing systems. So it's on its way. And there's so much we can do with that. We couldn't do otherwise because of the affordances of this technology. It is going to take a while to get all the right content and applications developed. We already have some really hard work done.

Good question. I think right now one of the challenges we have in our existing Healthcare System is that we don't leverage the power of community. I think there is a shift from having the clinic that not just is where you wait for a long time but where you see experts for a short amount of time and then you go home and you have questions. I think that's changing. But we haven't done this yet really. Our family, our neighbors are the people that were connected with through our academic work or school work, or a job. They can help us, too. I think we need to do a better job of supporting each other in terms of helping people with anxiety or stress, manage their stress. So, nothing I've done everything perfect. But I really like the way things have gone with me and my colleagues. I think what we all should have done in general as sink the personal computer revolution and then the social network and all the evolutions of technology. We need to do a better job of protecting people's privacy and finding ways for people to be present but not to have their data be harvested and used against them, to provide better agency for people that make it easy to control how your information is collected and used. There are companies out there making a huge amount of money off our information that we are getting to them for free and I think of that as of a way to better design the internet so that we could get micropayments every time we provided something and had more control of what we provided to these large companies that are harvesting data from us. I think the world would have been better. Individuals could have benefited, I think of musicians and artists and everyone who is contributing to our collective knowledge base will be rewarded. That basic architecture of the internet should have been designed so that we could support a way of controlling what information is shared. And also when people share information, to get them some financial remuneration based on how their data is exploited and used.

My first exposure to XR was as a concept. I might offend Jaron Lanier who is the person who coined the phrase «Virtual reality», and started the first commercial virtual reality company. We had a late-night meeting when we were just talking about things and I asked him what he was up to and he described his vision. For me, it was a really amazing revelation and it inspired me. It was going to be a better way for people to work with information, with computers and technology, and then also with each other. So, for me, that was probably 1985-1986 before Jaron had a chance to actually start building this medical app. We used to grab pieces together. We didn't have 3D interactive virtual environment at that time but we built them. And so for me, it was just so exciting to have that concept and realize the implications. Courses taken decades for all those implications and we're still figuring it out but it's been wonderful to see it evolve and actually become real.

That’s not fair to answer on one thing you could do!

I love that answer but I think that you're not doing something passive. You're not saying you want to watch a great performance but you want to be creative yourself. I think that is the power of these emerging technologies is that they're giving us back our latency. For too long things have been flat and two-dimensional. Now they are digital but still look to me as two-dimensional, a very narrow bandwidth of texting with fingers and thumbs. We've lost the richness of creativity, connection and communication. We've lost the nonverbal signals that we use to communicate with each other and convey emotional state. We've also lost ways to be creative with our hands and our singing voices. Some of us actually can sing. Not me. But the point is… I think we will know how to get to play point where we can be more creative and do it in a way that you feel comfortable with and express ourselves because it's hard to do that using other forms of VR technology. For a while this has been a very narrow pathway to express ourselves. And then again… I think that internet has not been claimed as a business model for creative people, too. So, I hope as the Metaverse emerge, you’ll get to do your performance again and again more than once. In general, I hope it's a platform where we can not only be healthier and happier, but we can be more connected to each other and help each other.
That was the point of trying to make it a little bit earlier. I think what would be really cool is if we in addition to looking better and looking out for our health care, will be looking at our friends. How are they doing? Are they in a bad mood? Or are they having a rough day or they struggling with something that is really difficult? Giving us the emotional support tools that we can use to help our friends or even neighbors and strangers to support everyone. One thing we can build into Metaverse is ways to understand each other better and ways to help each other better. I hope that's a part of design thinking that goes into the next generation of technology that we're all going to be living with for a while.