Featured Blog Posts

Latest Posts

Filter By Categories
September 29, 2016

Can AR and VR Improve Surgical Procedures Training?

We have discussed our primary research on augmented reality (AR) and virtual reality (VR) conducted in partnership with the Society for American Gastrointestinal and Endoscopic Surgeons (SAGES).

The purpose of the research was twofold: to gauge interest and adoption of augmented reality (AR) and virtual reality (VR) technologies and to explore the medical use cases for AR and VR – from the physicians’ perspective.

While not a scientific study the research uncovered and validated six medical use cases for AR and VR in medicine. This opens the door for healthcare product marketers and clinical educators to confidently introduce AR and VR into their marketing mix and go-to-market resources.

Today, we’re going to focus on the advantages and opportunities for the use of AR/VR in Surgical Procedures Training.

How AR and VR Improve Surgical Procedures Training

AR VR Surgical Procedures Training

Several doctors commented on the realism and immersive qualities that both zSpace and HTC Vive platforms create. The ability to recreate the surgical environment, instruments and disease states with high-quality 3D illustration conveys the sense of being a part of a procedure better than they expected compared to cadaver and animal labs.

The AR and VR platforms have a couple advantages over the labs.

  1. Alter the Anatomy On Demand
    You are not limited to anatomy that’s ‘on hand.’ With AR and VR, the 3D anatomy can be altered in any way you want.

For example, in these images, the user can drag a slider bar to change the size and shape of an aortic aneurysm.



  1. View from Any Orientation
    AR and VR simulations also give the user the ability to view the procedure and anatomy from any orientation.

For example, if you’re simulating a gall bladder removal and want to see how dissection of structures around the liver look from different angles, you can simply take a look at them from behind or below.

You can’t do that with a real gall bladder!

Required Improvements for AR and VR to Replace Animal Labs

Research participants also noted a couple of areas in which AR and VR need to improve to effectively replace animal labs in surgical procedures training.

  1. Improve Combined Views

One surgeon commented that they wanted to be able to see how the view they get in the operating room (OR) during a procedure corresponds to the photorealistic view they were seeing in the AR/VR application.

While not widely used in AR/VR interfaces today, it should be fairly easy to address in future versions of applications. A possible interface is shown in the image below where fluoroscopic views, similar to what a surgeon would see, are available at the same time as the ghosted photorealistic view. This would allow the learner to understand better what’s happening in the anatomy when they see it on a fluoroscopic view.


  1. Better Incorporate Feel

Currently, AR and VR platforms don’t handle haptic feedback well enough to meet clinical expectations.

While surgeons found AR and VR most useful for procedures that they couldn’t visualize directly, those are also the procedures in which the ability to ‘feel’ the interaction between devices and anatomy are most important.

Platforms such as HTC Vive and Oculus Rift do have add-ons that can track a surgeon’s hand or a device. Combining tracking with 3D printing of practice anatomy/disease states may provide a way to overcome haptic feedback issues in relatively inexpensive fashion.

Where AR and VR Are A Natural Fit

Because of the level of immersion available in AR and VR, it is generally accepted that both could be used for medical training.

Intravascular procedures, radiology, hernia repair, gall bladder removal, colorectal and upper gastrointestinal surgery where highlighted by the surgeons as ideal training opportunities.

Specific suggested uses centered around surgical procedures in which:

  • The structures being operated on couldn’t be directly observed;
  • Where intricate dissection of nearby structures was required; and/or,
  • Situations that require blind movement of instruments.

Surgeons affirmed that AR and VR would be useful as simulations to practice the procedures and were most appropriate for medical students, interns and residents.  And the use of 3D print augmented reality could also introduce haptic feedback for simulations.

Economic & Ethical Advantages of AR and VR for Surgical Procedures Training

There are economic and ethical issues in using animal and cadaver labs that can be mitigated by using AR and VR applications in simulations.

Considering the cost of animal labs has been pegged at between $50,000 - $72,000 per resident per year, the cost of a high-end VR system near $3,000 can make a lot of financial sense.[1],[2]

Further, the ethical concerns of operating on animals are becoming acute. According to a 2008 study, an increasing number of surgeons are refusing to practice on animals, even if it hurts their careers.[3] So the challenge for those of us creating AR and VR tools, is to make and validate sufficient tools to render the use of animals for practice, obsolete.

Discover the Six Medical Uses Cases for AR and VR

Using AR and VR to change and improve how surgical procedures training is done in modern medical practices, is but one of six use cases the research uncovered.

[1] Bridges M, Diamond DL. The financial impact of teaching surgical residents in the operating room. Am J Surg. 1999;177:28–32.

[2] Rehman S. et al. Simulation-based robot-assisted surgical training: A health economic evaluation. Int Jour Surg. 2013; 11:841-846.

[3] Collier R. Switching swine for simulators: At what cost? CMAJ. 2008; vol. 179 no.8: 759-760.


Subscribe to Our Email