Metaverse - Sergey Doronichev

Sergey defined Schlumberger’s product strategy for AR/VR apps. Launched several XR products. Managed execution across multiple SWE sites, stakeholder/customer/cross-function communication, and updates to C-level management.
• Led product development, UX workshops, and broad exploration of AR/VR use-cases, resulting in 75+ prototypes, 5 patents, and 8 launched products.
• Developed and launched “FracTetris”, an industrial product for outdoors layout planning using AR, VR, and drone tech. It has been used on more than 300 jobs in the first two years reducing the non-productive time by 80%.
• Launched a company-wide training program in VR. Developed a prototype, got funding, and scaled deployment to 40 VR classrooms worldwide. Currently training hundreds of external and internal students per year.
• Initiated and scaled adoption of AR glasses by ops teams in 7 countries leading to +50% efficiency for major use-cases.
• Built a tech community with 1000+ active members. Organized international in-person and virtual tech conferences.
• Organized a pipeline of media content creation, including blog posts, promo videos, and interviews with the tech leaders.
• Acted as a liaison for the tech industry, sourcing and signing partnership deals, evaluating investments, and attracting talent.

As a Product Manager of augmented & virtual reality projects here at Schlumberger I work from the start basically on everything related to XR. We work on three ways I would say. For virtual reality, we work on the creation of virtual reality trainings. We’re trying to apply VR for other collaborative scenarios like conducting VR meetings. Recently we started to use VR workrooms developed by Meta. That’s really incredible. Actually, we have our daily startup meetings in VR now with my team.
We work on the development of products for scientific visualizations when people across the globe can join in the same instance and collaborate together around some specific types of data. Our case is mostly geological, physics, or engineering data. That is really complex stuff. But the important thing is that in VR you can do it much better and much more efficiently compared to Zoom meetings. We are having on Smart Glasses and deploy ‘em worldwide. It’s like a Google Glass stuff but these we are aware because we are more industrialized. Sure, we do some development for mixed reality. This is really incredible. There are a bunch of use-cases that you can do for mixed reality as well!

Sure! I moved to California almost 6 years ago at the beginning of 2016. If you remember that time (particularly in February) it was when several companies – Facebook, HTC, Vibe, Microsoft – released VR/MR headsets, right? HTC Vibe on their own, Oculus Rift & Microsoft HoloLens. Everyone here in Silicon Valley wanted to talk about VR and I was lucky to get one of the headsets here in my possession. And I’ve just fallen completely in love with it! Everything that we read about in childhood or watched movies appeared to be so real and it was already there.
Basically, I started to develop some quick prototypes on my own. I started to learn Unity in my free time. Quite quickly I realized that there is a lot of potential for enterprises in that. So I kept up with the strategy for VR/AR to be used in enterprises internally for our executives, got funded, and started my own pillar at the company. But now we have VR/AR projects across the globe and several teams is working on that.

I thought like FINALLY! Finally, this industry is moving in another step of evolution for as I mentioned I’ve been working on this since 2016. And until 2018 it was a really small niche of people who were using it and trying to apply it. But now when such companies as Meta or Microsoft start investing billions of dollars in this and thousands of people working on this stuff I’m quite positive that very soon this will become much more adapted than it was in the past. So I was very excited to hear that news.

First of all, I would really like to define the term Metaverse because different people mean different things by saying that. And it’s not like right or wrong! I just want to clarify what I mean by Metaverse. For me, Metaverse is not just a single 3D World that exists somewhere. It is more like a concept of how people will interact with digital information in the future for me. Instead of looking at this digital stuff on a 2D screen we will become immersed in it and will be looking at the Internet inside of it. This is what I mean by Metaverse.
In this context, it’s not really what only Facebook builds, right? This can be anything. Many other companies will use this term for a while. Actually, if we go back then this term was introduced back in 1992 in one of the book novels. People who worked in VR already heard it for a while, right? So, It’s not just what Mark Zuckerberg invented right here. As for what we can expect from Facebook or Meta no one knows. I do believe that even Meta itself doesn’t know what they can build! But they already introduced several products in that field. They have Horizon workrooms, worlds, and Horizon Advanced that allows you to have conferences, small concerts, or whatever. But I haven’t really tried the last one yet. Of what to expect from Facebook it will be a massive multiplayer social experience in VR.

What I get is that it is important to understand what we mean by Web 3.0 because for many people Internet is decentralized but for others, it is much more when you structure permissions differently. Definitely, Metaverse will be a part of our lives and I’m quite sure that it will be a part of the Internet in whatever way we will have it. If it’s going to be an interface that people use to access the data or the web then sure this will be a crucial part of it. But whatever we think now I’m quite sure that in 5 years it will be in a different form than we imagine it today. You know how it happens then you start building stuff and then you just realize that some things don’t work while others just pop up and you realize that those are the coolest ideas you have ever seen.

There are lots of them, actually. XR by itself is the first one. Hardware if it becomes smaller and when it becomes smaller & easier to use. When we’ll get more standalone headsets like Oculus Quest and this will become more accessible to other people. Right now not everyone actually wants to put something on their head, especially during the workday. Second is that not a lot of people could stay in VR still for a long time so it’s more comfortable for most of us to stay for like an hour or two maximum. But then you just get tired. Any technology related to hardware that will help to make those experiences smoother will benefit the creation of Metaverse.
Then it’s the concept. Right now you really have to have a reason to get in VR. I already do it because I work in that with development stuff. Or you may be a gamer or a super early adapter of technology saying - ‘Wow, it’s so cool. I’ll put this headset on and I will go & play that 15 minutes experience’. But there a not that many AAA games yet. Because many companies realize that VR is probably coming but not today. And it’s definitely not a cash cow for them! In such terms it’s still a wild west here for people who want to develop high-quality content are welcome here. So right now they do have really good chances to build companies around that and have their own successful products.
Also we got social experience here. Everything related to communication in VR and meetings these days the biggest VR multiplayer experience we can have will be like 20 people in the same room. But 30 years ago we couldn’t imagine that you will have thousands of users playing WoW or some others via the Internet. Today you have it and take it for granted so I believe this will be developing as well and in 2 years we will have truly massive multiplayer experiences.
Moreover, there are many things people talk about right now. We’re talking about virtual economies, on how people will trade items there and spent there more time. Your look will become really important for you. We have some startups like a dress for example. There are platforms that allow you to buy & sell digital clothes, dresses, hats, and appearances. Either you can create similar tools for other things, similar platforms or you can actually start creating content. Content creation is another future topic here because if we want to create really amazing immersive worlds then someone needs to create 3D assets for them. But today it’s not that super easy so you need to be educated and take a class at least in 3D design to build something. There are some opportunities to build tools everyone can use to build whatever they want or just become a 3D artist to create some stuff by yourself.

I think that it’s really important here to separate two types of scenarios for business-related use-cases. I mean not VR-related business – enterprises & consumers only! Enterprises like ours… We’ve been using VR & AR for like 5 years quite heavily for various use-cases I already mentioned. You can use augmented reality when trying to fix something. I’m not sure for a company but feels like Ford recently announced that one of the carmakers here in the USA deploy a couple of thousands of Smart Glasses for dealerships because it will help frontline workers do jobs better. So they will have that tiny piece of glass in front of their eyes… And first of all, they can do as you mentioned FaceTime type of calls… Or it can provide you with standard work instruction. In other words, you use your voice to see information related to the work you do right now. For example, you want to change the wheel and it gives you step-by-step guidance on how to do that. So for enterprises, there are lots of use-cases. People who are involved in that know about it and already use it.
Speaking of consumers is much more interesting because if we’re talking about VR gaming and some social experiences especially now in COVID that speeds up this whole remote work… Remote social experiences are developing really fast now. You may want to meet friends, play VR games together, watch movies or sports events together… You can create things together in 3D VR.
There are other use-cases that I believe will be very popular in nearest future like AR navigation and in-street performances. Imagine you are a tourist coming to a new city putting smart glasses on your head… Then you walk through the city and it highlights you all the information you need related to the sights you wish to see. Also, you may come up with some new ads experiences. You may go to some street corner and see Cristiano Ronaldo doing some crazy tricks there or watch street dancers. Or you may see graffiti on the walls that some people draw on real historical buildings. However, these AR experiences are tied to hardware limitations today. You may not have powerful smart glasses that are easy to wear like those Ray-Ban things on the street. Ray-Ban is not an augmented reality for they just have a camera.
Another thing is that an augmented reality works outdoors much worse compared to indoors. You can have a very comfortable experience when you’re indoors but when you have sunlight most of today's devices don’t really allow getting a comfortable mixed reality experience. So there are technical issues to be solved together.

I briefly mentioned in the beginning that we need a reason to use VR today. This means that as a community we have to provide users with those needs developing great content, social experiences that weren’t possible in the past. I think this is the key. Second thing is that we’ll probably be some kind of merge between VR and AR experiences or I would say cross platform experiences because even if you get today into some multiplayer activities in VR then most likely you’ll play or communicate with other users who ought to have VR devices. What we found extremely beneficial in enterprise environment is that if allow VR users joining same sessions together with people who don’t have VR devices. So the more content will be created like that.
Imagine one person plays Battlefield or Counter Strike by using desktop and another one is joining that same game with VR device. This will stop limiting VR community just by people who got VR devices. This is actually evolving but it takes time. I don’t really see some roadblocks that we need to overcome with some crazy ideas how to fix those problems. We’ll get there. I think it needs time.

I think that it exists already and it is called an Internet. If coming back to the Metaverse term this is just an interface, a way to interact with the data that exists in seamless worlds. I don’t think that we’ll have one single Metaverse 3D World that you may join from your VR or mixed reality devices all the time. But I’m going to rephrase the question to when we’ll see massive adoption of it? When we’ll be time when people will just use their VR or AR headsets as often as they use mobile phones today? I think we’ll get there really quickly in 10 years or so. Right now we’ll see a new way of hype on this topic. Then we’ll see a new wave of investments in XR industry. Companies will start building more products, new devices, new experiences, and we’ll see results of that in next 3-5 years.
I would say that in 5 years you’ll have lots of content and it’ll really difficult to meet a person who didn’t try VR in own life. So my vision is that we need 5-10 years to get there.

Facebook connects people. Now they are planning to do the same thing just slightly different. Definitely, Facebook is some kind of a “verse” – Universe, Metaverse or Facebook-verse. Again! It’s about social connections, sharing experience. Facebook plans to do the same thing just slightly different. That’s why I’ll probably agree with that statement.

I do really think that’s really important point you mentioned. First of all, everything you described and most of those things you can do with Facebook as well. They have Facebook Watch and 30-50 products inside that are able a lot to do and all that crazy stuff – a Facebook place. All that spaces & places… They basically have armor for engines, they got enterprise solutions. You can work and chat with your friends in Facebook, you can watch movies together. They have Facebook live rooms, clubhouse experiences and so on. Other people will not use Google maps but they can use AR or maps provided by Apple doing same things never leaving that ecosystem built by Apple. It really depends on what type of user you are!
What is really interesting here is that will we have same division in future in terms of Metaverse? Will we have some Metaverse built specifically for Apple users? Everyone heard rumors that company is working on Smart Glasses. No one has ever seen them but I believe we will get them one day. Will those companies try to limit those experiences for their users and consumers? They may allow you to transfer your 3D assets. For example, how your avatars look or whatever… 3D-pad that you have from one Metaverse to another…
But how this transition will be happening? Will those companies want that transitions? Or will they try to completely isolate that experience so that users will never leave their ecosystem? That would be very interesting!
I heard that interview of one of Facebook’s executives. He said that today they are basically in the same world. And today it’s their common interest to actually develop the entire thing right. If this integration will allow them getting there faster than they probably will not try to limit those experiences specifically like I mentioned by organizations. But there can be lots of speculations on that topic. I think anyone doesn’t know now how it will look like.

It’s a very tricky question! Who should regulate it in a real-life? We still don’t have an answer because everyone has a different opinion on that. In real life, we have a kind of system that somehow works. We have a government that controls all those things. Governments can be corrupted, some people can make mistakes and every time there will be someone not happy with the decisions. But this is the tool that somehow works.
What will be happening in Metaverse and who should do that? Honestly, I don’t know. I don’t know a better controlling system in a real-life that exists today! Definitely, those are topics that people discuss right now, especially if we go to a decentralized Internet where you don’t really have some entities that control experience, content, and so on. This way we can get some nasty places. For example, if you use Facebook today… Facebook has your data. They know users’ age. They can control when you get kids and can provide appropriate content for them. All that is because they have this information. If you stop controlling those experiences then who will do that? How can you be ensured that your kid goes to some Facebook Horizon experience that is not designed for kids? That’s why it’s a tricky question. I don’t think anyone knows the answer. I believe that for a while companies will continue to do what they are doing right now. So this will be kind of a mixture of government, policies, and enterprise policies that are normally much stricter than in the government’s world just to keep it safe. We’ll see…

I think quite soon we will get there because, for example, the gaming industry normally pushes those things harder. Today you have such solutions as Photon – a party service that most multiplayer games use. Not most but many just to synchronize users’ experiences. I know that companies already support VR experiences. Sometimes we use it for ours. The number of users has already increased and increased for the last year and will continue to grow. I think within 3 years we will get truly massive VR social experiences like concerts or games.

Why not? That’s funny! Just yesterday we had similar discussion with my friend from Apple. What are pros & cons of open environment like Google for example and of much more isolated environment like Apple? For those who don’t know about it… In Apple there are parallel teams that don’t know what each is working on but at Meta everyone knows of all projects and can access most of the data. When you have that opened environment then it really allows you to leverage other people’s experiences and don’t make similar mistakes. In my opinion it’s much more beneficial for the entire thing – for company and for a Metaverse as a concept. If people start sharing the code then this will help other people build new experiences much faster.
On other hand, we need to be sure that people who come up with these ideas and develop solutions have proper motivation to continue doing that because when you make an open-sourced solution you can make money out of that. Then why you can come up with something crazy when you actually need to feed your family? You cannot work on something else in such cases, right?
Overall I believe that we’ll have some kind of a balance and most of things will be open-sourced. This will allow us enjoying Metaverse is a much faster & better way.

I’m sure that our expectations will never be met! On other hand, what we have today is that everyone can choose Metaverse they like. Some like playing Counter-Strike but others prefer being a knight and playing WOW! That depends on what you imagine and want. People will be able to enjoy their experiences. So that fantastic movie… I’m quite positive that people will come up with some kind of a Ready Player One movie version! Some will crave for Matrix experience. That depends on how well it goes. Millions of people will be enjoying that.
Same thing today! Some people just leave Fortnite but some play all the time. And they are not playing all night. They just communicate with friends there discussing homework or just ways of how they interact today. But everyone will choose their own Metaverse version.

That one is easy. Definitely, Meta & Microsoft. They kind of splitting the market on player and consumer experiences related to XR. Microsoft is the biggest player in the enterprise XR market even though they are focused more on mixed reality experiences, on the whole ecosystem they build, and on the integration of today’s Microsoft products which are hundreds! This makes this company the biggest player in the enterprise VR/AR market.
Also, there are some companies I already mentioned like Fortnite… It’s not a company but a product. But we have Roblox and Minecraft. Even though they are not VR/AR experiences but they do satisfy most of the features that Metaverse should have. So you have lots of worlds that exist whether you are online right now or not but the presentation of yourself is there and exists. You also have virtual economies and so on.
I think that those are the biggest players today and we’ll see if they move towards AR/VR in the future!

In XR? I suppose in my case it’ll be something related to meaningful social connections that you are unable to reach right now. This would be great to have something that would allow us to integrate all those neurobiology things like neuro link.
Imagine a person who is completely paralyzed and unable to talk. But somehow due to microchips in his head you will give a chance to those people to have their digital representation in a Metaverse where they’ll have full body capabilities, could walk, talk, live normal lives in Metaverse in XR while unable to do that in real life. If I had to choose one then I believe this to be that very use-case. I would give people a chance to live in the virtual world and talk to those they love while unable to do that in real life.

I can tell you about my project then. We were solving a task for field operations where you have 50 huge trucks that you need to position for some jobs. Normally we need a few days just to park that trucks correctly and install equipment accordingly. But sometimes if you make a tiny mistake in planning then imagine you spent a day or 24 hours positioning those trucks and when the 36th truck is coming there is not enough room for a turn and you have to re-park trucks wasting another few days just to do that. So we created a platform solution that integrates drones, VR & AR. And now we have a drone that flies around this spot mapping all in 3D in just a couple of minutes.
When getting this 3D environment on the VR platform people can join these collaborative experiences from multiple devices, VR headsets, tablets, and laptops. There they create a 3D plan of a whole simulation positioning equipment in the right places and running those trucks to see how the environment evolves all the time to ensure that all steps are going smoothly. When approving that, they just press the button and the whole thing goes to people in the field who use iPad to see an augmented reality representation of this digital field and all this equipment in the real world. And then when those trucks are coming they just coordinate & position them in a way for a real truck to overlap its own 3D representation. It was super successful and helped to significantly reduce the number of errors and non-productive time.
Another thing I really love in XR and have been talking about for a while is a Tesla suit. Have you tried those, guys? Those suits allow you to feel everything that is happening in VR! It’s again some kind of a Ready Player One & Matrix concept! But those are already available today! Actually, I do have a couple of them in my office here because we use them for VR training. We don’t use them on daily basis for VR training but we use them to approve the concept. So in training, we have explosions or some other things that affect your body and you actually feel them and understand what you should do in real life or whatnot.