The purpose of the research was twofold: to gauge interest and adoption of augmented reality (AR) and virtual reality (VR) technologies and to explore the medical use cases for AR and VR – from the physicians’ perspective.
While not a scientific study the research uncovered and validated six medical use cases for AR and VR in medicine. This opens the door for healthcare product marketers and clinical educators to confidently introduce AR and VR into their marketing mix and go-to-market resources.
Today, we’re going to focus on the advantages and opportunities for the use of AR/VR in Surgical Procedures Training.
Several doctors commented on the realism and immersive qualities that both zSpace and HTC Vive platforms create. The ability to recreate the surgical environment, instruments and disease states with high-quality 3D illustration conveys the sense of being a part of a procedure better than they expected compared to cadaver and animal labs.
The AR and VR platforms have a couple advantages over the labs.
For example, in these images, the user can drag a slider bar to change the size and shape of an aortic aneurysm.
For example, if you’re simulating a gall bladder removal and want to see how dissection of structures around the liver look from different angles, you can simply take a look at them from behind or below.
You can’t do that with a real gall bladder!
Research participants also noted a couple of areas in which AR and VR need to improve to effectively replace animal labs in surgical procedures training.
One surgeon commented that they wanted to be able to see how the view they get in the operating room (OR) during a procedure corresponds to the photorealistic view they were seeing in the AR/VR application.
While not widely used in AR/VR interfaces today, it should be fairly easy to address in future versions of applications. A possible interface is shown in the image below where fluoroscopic views, similar to what a surgeon would see, are available at the same time as the ghosted photorealistic view. This would allow the learner to understand better what’s happening in the anatomy when they see it on a fluoroscopic view.
Currently, AR and VR platforms don’t handle haptic feedback well enough to meet clinical expectations.
While surgeons found AR and VR most useful for procedures that they couldn’t visualize directly, those are also the procedures in which the ability to ‘feel’ the interaction between devices and anatomy are most important.
Platforms such as HTC Vive and Oculus Rift do have add-ons that can track a surgeon’s hand or a device. Combining tracking with 3D printing of practice anatomy/disease states may provide a way to overcome haptic feedback issues in relatively inexpensive fashion.
Because of the level of immersion available in AR and VR, it is generally accepted that both could be used for medical training.
Intravascular procedures, radiology, hernia repair, gall bladder removal, colorectal and upper gastrointestinal surgery where highlighted by the surgeons as ideal training opportunities.
Specific suggested uses centered around surgical procedures in which:
Surgeons affirmed that AR and VR would be useful as simulations to practice the procedures and were most appropriate for medical students, interns and residents. And the use of 3D print augmented reality could also introduce haptic feedback for simulations.
There are economic and ethical issues in using animal and cadaver labs that can be mitigated by using AR and VR applications in simulations.
Further, the ethical concerns of operating on animals are becoming acute. According to a 2008 study, an increasing number of surgeons are refusing to practice on animals, even if it hurts their careers. So the challenge for those of us creating AR and VR tools, is to make and validate sufficient tools to render the use of animals for practice, obsolete.
Using AR and VR to change and improve how surgical procedures training is done in modern medical practices, is but one of six use cases the research uncovered.
 Bridges M, Diamond DL. The financial impact of teaching surgical residents in the operating room. Am J Surg. 1999;177:28–32.
 Rehman S. et al. Simulation-based robot-assisted surgical training: A health economic evaluation. Int Jour Surg. 2013; 11:841-846.
 Collier R. Switching swine for simulators: At what cost? CMAJ. 2008; vol. 179 no.8: 759-760.