Virtually Better

The COVID-19 pandemic couldn't have come at a better time for virtual reality.  It has caused many workers to work remotely, introducing many workers to collaborative tools like Slack or Microsoft Teams and even more to video platforms like Zoom or Skype.  But we're just beginning to understand what collaboration could look like -- such as virtual reality (VR).

As CNBC noted: "Virtual reality is booming in the workplace amid the pandemic."  Even a pre-pandemic Perkins Coie survey, done for the XR Association, predicted an explosion of immersive technologies like VR, augmented reality (AR), and mixed reality (MR).   Elizabeth Hyman, President of XRA, said: "We are at the precipice of an integration of XR technology that will transform businesses and society for the better.”  

The report expected healthcare to be the industry most impacted by immersive technologies (outside of gaming/entertainment).

Spatial virtual meeting.  Credit: Spatial

Take VR-start-up Spatial, which thinks it has a better mousetrap.  Chief Product Officer described their solution to MIT News:

Spatial is a collaborative, holographic, augmented reality solution.  You can teleport to someone’s space, work as an avatar sharing that 3D space, and use it instead of a screen to manage a project, present an idea, and more.

Don't you love the "and more," as though the teleportation wasn't enough? 

Spatial has been around since 2016, but in the wake of the pandemic Spatial made its enterprise version of the platform free.  It works on most virtual reality headsets, although there is a web browser version that can be used without a headset.  Design and engineering companies have been the biggest users, but that is changing.  "After coronavirus, interest went up 1,000 percent," Dr. Lee said, "and a big part of that was from smaller businesses, hospitals, schools, and individuals."  

In the shared workspace, people -- or rather, their avatars -- can use whiteboards, share sticky notes, build and use 3D models, even give each other high fives.  Here's a short promotional video:

Dr. Lee points out:

There’s a lot of Zoom fatigue right now, and I think the biggest reason why is because the video format really forces you to be 200 percent focused when you’re presenting or listening, but you can’t do something together.  You can’t be in this space, looking at things together and pointing at things. This feeling that we’re in the same space can only be achieved through a 3D physical office, and that’s some of what Spatial is trying to achieve in virtual form.

Microsoft, for one, is not sitting idly.  It just introduced a "Together Mode" to its Teams platform, which "uses AI segmentation technology to digitally place participants in a shared background, making it feel like you’re sitting in the same room with everyone else in the meeting or class."  It is supposed to allow participants to pick up on faces, body language, and other non-verbal cues.

Here's Microsoft's promotional video:

Marissa Salazar, Product Marketing Manager, told TechCrunch's Frederic Lardinois: "you’ll notice the way that we’re looking at each other is obviously very different than something we’re used to, not only are we out of the grid, but we’re looking at this, ‘mirror image’ of ourselves."  Microsoft's research suggests that, based on monitoring brain activity, participants using the Together Mode exerted less mental effort than in the traditional grid mode, thus reducing meeting fatigue. 

Currently, the shared space is an auditorium, but additional views are expected to be available soon (including a coffee shop for smaller meetings).  Famed VR pioneer Jaron Lanier, who worked with Microsoft on the new tools predicts: “We’ll see people make the space their own, just like they do the workplace, for the type of social dynamics that may exist.  I think where this is headed is a lot more flexibility and customization.”

At the same time, Teams is introducing Dynamic View to "dynamically optimize shared content," allowing more control over how users see content and other participants.    

Mr. Lanier says: "In a sense, it’s just a simple design strategy.  In another sense  it’s a design strategy that benefits from many years of studying mutual perception, particularly in virtual reality."  

Take a look, for example, of what Imperial College (London) claims is the first virtual ward round for medical students, using HoloLens.  The attending physician sees the patient in person, and streams the visit to literally hundreds of medical students.  The Guardian reports: "Teachers are able to pin virtual pictures to the display, such as X-rays, drug charts or radiographs, or draw lines to highlight something they want to emphasise."  

One medical student gushed: 

Despite the ward round being virtual, it felt far from it – we were expected to ask questions and think about clinical problems in real time.  It was really helpful to be able to access investigations like X-rays and blood tests in an instant, and the way the information was projected felt natural.

Now imagine what that experience could be like using Spatial. 

Last year I suggested that EHRs needed a new metaphor; instead of just being digital versions of paper records, they should be collaborative tools like Slack or Teams.  Now I'll go further: they should be a "collaborative, holographic, augmented reality solution,"  Oh, wait, that's Dr. Lee's description of Spatial's solution. 

Spatial's slogan is "How Work Should Be," but it's not how healthcare is.  Healthcare is full of telephone calls, long waits, 1-on-1 meetings, incomplete or incomprehensible data, even faxes.  It is a maze we navigate anxiously, and sometimes get lost in.

For many, perhaps even most of us, our healthcare journey is a team effort, not just with our healthcare professionals but also our support system.  We need better ways to visibly present, interpret, and use our data, and to get that data to the right people at the right time. 

We need XR healthcare.

I don't want to live in the metaverse (as Neal Stephenson termed it thirty years ago) and I certainly don't want to live in our current pandemic world any longer than I need to, but the pandemic is helping us discover new ways to do things.  The shared, collaborative world that companies like Spatial or Microsoft are working towards is one that I hope we keep, especially in healthcare.