A custom virtual reality training solution for ophthalmologic surgical clinical trials
Advances in Simulation volume 6, Article number: 12 (2021)
We present a summary of the development and clinical use of two custom designed high-fidelity virtual-reality simulator training platforms. This simulator development program began in 2016 to support the phase III clinical trial Archway (ClinicalTrials.gov identifier, NCT03677934) intended to evaluate the Port Delivery System (PDS) developed by Genentech Inc. and has also been used to support additional clinical trials. The two simulators address two specific ophthalmic surgical procedures required for the successful use of PDS and provide state-of-the-art physical simulation models and graphics. The simulators incorporate customized active haptic feedback input devices that approximate different hand pieces including a custom hand piece specifically designed for PDS implantation. We further describe the specific challenges of the procedure and the development of corresponding training strategies realized within the simulation platform.
The industry development of simulators in healthcare is generally focused on such surgical, procedural, or diagnostic challenges that occur with high volume and are mostly performed according to well established and possibly international standards. This combination has the best chance of producing a profitable business case which can carry the extensive costs usually associated with the development of a new high-fidelity simulation solution, especially when virtual reality (VR) technologies are involved. Consequently, the bulk of available simulator products target a rather small set of medical procedures which is particularly true for surgical simulators . Custom development commissioned by industry partners from pharmaceutical and medical device industries constitute a segment of the surgical simulation market that extends the range of simulator solutions to product-specific and non-standardized procedures but remains limited to high revenue products to justify the investment.
The partnership formed between Genentech Inc. and VRmagic GmbH in the autumn 2016 deviated from this pattern in that it was formed to produce a training device for a product that was still in development and had not yet been commercialized. Genentech was about to start a phase II study for a new and innovative product, the Port Delivery System with ranibizumab (PDS), which is a new permanent and refillable eye implant, approximately the size of a grain of rice, that releases drug continuously into the vitreous of the human eye. The phase II Ladder study (ClinicalTrials.gov identifier, NCT02510794), specifically evaluated the effects of a specialized formulation of ranibizumab delivery for the treatment of neovascular (wet) age-related macular degeneration (see ). The initial clinical experience with the implant had indicated that the implant insertion can be performed safely as an outpatient surgical procedure. However, the implant insertion surgery involves a number of surgical steps not commonly performed by vitreo-retinal surgeons. Furthermore, the refill-exchange procedure that exchanges the implant content with fresh drug only resembles the commonly performed intravitreal injections but requires a more precise execution. To ensure patient safety and minimize the risk of surgical errors to the overall outcome of the trial, Genentech’s development team started looking for the best possible way to train the study investigators on the PDS procedures. After 2 years of intense development efforts, in the spring of 2018, Genentech and VRmagic rolled out two completely new, state-of-the-art virtual reality simulator platforms enabling its users to practice the implant insertion and the refill-exchange of the PDS, not in support of training and marketing for an existing product but as a training aid for surgeons involved in ongoing PDS clinical development.
While some training data has already been collected, including performance metrics and training meta-data, previous training sessions were focused on providing the best possible training of the trial investigators. However, protocols that would ensure an unbiased evaluation of the simulator itself were not a priority. As it is currently unclear when statistically relevant data suitable for the validation of the simulator construct might be available, this article intends to provide a concise description of the hardware and software that constitutes the simulator platform.
The list of distinct AR/VR surgery simulator systems which are consistently employed in real-life training programs is short ( lists 70 types of simulators over all medical fields that were developed during the last 20 years) and grows shorter if we consider only systems with active haptic feedback .
The insertion procedure of the PDS implant is illustrated in Fig. 1. First, dissection of conjunctiva and Tenon’s capsule at the implant site in the superotemporal quadrant of the eye is performed. This is followed by a precise scleral dissection to expose the underlying pars plana. The vascular tissue of the pars plana tissue gets then coagulated with an ab-externo laser ablation technique  to mitigate vitreous hemorrhage from pars plana bleeding, a potential complication of vitreoretinal surgical procedures that generally result in vision loss .
For the refill-exchange procedure, the surgeon employs a syringe with a custom needle that includes a reflux chamber and exchanges the contents of the implant with the new drug in the syringe simultaneously. A successful refill, as illustrated in Fig. 2, requires the surgeon to insert the refill needle perpendicularly into the implant septum. With a septum diameter of about 1 mm, a complete insertion requires both a central entry point as well as orthogonal alignment of septum and needle.
The simulator platforms
Although their ergonomic layouts are distinctly different, the two simulator hardware platforms shown in Fig. 3 were designed with the intent of maximizing the set of shared components: both platforms are mounted on a mobile base unit that contains the computing hardware and in both cases active force feedback devices were used to capture the user input and render haptic feedback. As the development timeline did not allow the development of a custom force feedback system, both platforms employ the TouchX devices produced by 3D Systems. When bought off the shelf, the device comes with a fixed stylus that is held by the user like a pen and attached to the actuated device arms via a gimbal. However, the design does not enable the user to exchange the stylus with a custom handpiece and the extent of the gimbal mechanics prohibit the usage of two devices in sufficient proximity to simulate user input during the implant insertion procedure. This specific procedure requires bimanual instrument interaction on a surgical field spanning less than one octant of a human eye. To avoid device collisions during the simulation, the gimbal and stylus parts of the device were substituted by custom components designed by the VRmagic development team with a geometric configuration optimized for ophthalmic surgery training. As the procedures require the use of different instruments that cannot be accurately represented by the same physical handpiece, the new gimbal construction was augmented to allow exchanging handpieces that can be fastened via a magnetic lock system and communicate internal degrees of freedom, e.g., the opening angle of a forceps, via spring loaded connectors. In total, four different handpieces were developed. Their shapes and internal degrees of freedom were modeled to emulate (i) forceps, (ii) a syringe, (iii) a number of simple straight instruments (e.g., scalpel or cotton tip) and (iv) a special tool designed by Genentech for the sole purpose of placing and releasing the implant.
The salient differences between the two platforms presented in Fig. 3 arise from the distinctly different contexts in which both procedures are performed. While insertion of the implant requires the surgeon to operate in an adequately prepared and staffed OR environment, the refill-exchange can be performed by a single practitioner with minimal preparation in a clinical office setting. The former requires the surgeon to sit and observe the situs through a microscope, the latter is to be performed in a standing position, slightly bent over the patient sitting in an ophthalmic chair and using only loupes for magnification. The differences in the ergonomic setup translates into different work space requirements on the haptic input devices and correspondingly different positions and orientations of these devices relative to the virtual situs which constitute the predominant constraints for the platform design.
For the implant insertion procedure, all visual feedback from the virtual scene is generated by a special stereoscopic viewing system. It was developed by VRmagic in cooperation with Haag-Streit Diagnostics and combines an actual microscope lens tube with two micro displays providing a resolution of 1080 × 1080 pixels per eye.
The development timeline prohibited any attempt of developing a custom head mounted display that would optimally simulate the loupes provided to the investigators for the refill-exchange procedure. However, by modeling the optical effects of the magnifying lenses within the graphic rendering pipeline, the requirements on the actual hardware displays could be sufficiently reduced to allow the integration of a consumer product. After weighing the benefits of the various products available by the beginning of 2018, the Oculus Rift CV1 was selected due to the high fidelity and stability of its tracking system.
Simulator training benefits
Today, there are various methods for simulation-based surgical training in ophthalmology including traditional wet lab training with animal models or hybrid animal/synthetic models, dry lab training on synthetic models as well as virtual reality training simulators . While clinical development is still ongoing and there is not yet any data that would allow for a validation study of the simulator training, the following descriptions of some aspects of the training modules provide examples to illustrate the potential of virtual reality simulators to teach these procedures efficiently.
For the implant insertion procedure, the simulator curriculum covers only the parts with the highest impact on the clinical outcome starting with the scleral dissection and ending with the actual insertion and placement of the PDS implant. This partial procedure is split into four separate training modules comprising (i) the scleral dissection, (ii) the laser ablation of the pars plana, (iii) the pars plana incision, and (iv) the actual implant insertion.
For the refill-exchange procedure, a separation into more than one training module is not appropriate. Instead, the procedure can be performed on different virtual patients representing varying levels of difficulty essentially controlled by the visibility of the implant through the conjunctiva and the haptic resistance during insertion.
Each training module can be started directly thus enabling the user to train procedural steps out of order and focus their efforts on those tasks considered most challenging. Furthermore, the user can restart a module at any point and thus reset the scene for a new attempt instantaneously. Out of order training and immediate restarts provide a significant benefit that cannot be attained when training on animal models and constitute crucial features considering that training sessions for the trial investigators may last only 15–30 min.
The PDS implant needs to be seated within a scleral incision of a very specific target length that corresponds to the long axis of the implant, with a minimal margin for error. Furthermore, the incision needs to be created without harming the underlying pars plana tissue. Considering that the average thickness of the scleral tissue is only ~ 0.7 mm , the surgeon needs to navigate his blade within a very narrow corridor while compensating for the erratic forces resulting from the toughness of the scleral tissue.
To enable a real-time simulation of this process, the simulator employs a very fine and dynamic sub-triangulation of the scleral tissue adjacent to the expanding cut. The resulting resolution is sufficient for an accurate physical simulation model and the rendering of consistent haptic feedback. A significant effort was made to obtain an accurate graphical rendering of the fibrous scleral tissue allowing the user to learn the correct interpretation of visual cues that indicate the incision depth within the scleral tissue.
Compared with other conceivable training methods, a major benefit of this approach is the ability of the VR simulator to compute and report the relevant metrics in real-time as well as in a posteriori analysis. The former is provided as a heads-up display (HUD) that allows the user to directly associate subtle visual cues with objective depth information and thus improve their ability to assess the incision geometry (see the first picture in Fig. 4). The HUD further allows the user to recognize common mistakes like rounded depth profiles at the incision corners which are hard to recognize even by expert observers. A posteriori evaluation of user performance includes information about tissue treatment as well as the latitudinal and longitudinal placement of the incision.
Laser ablation of the pars plana
The fundamental challenge of this procedure is the correct application of the laser probe which is used in a context and configuration not usually employed in vitreoretinal surgery. The ablation of the pars plana tissue needs to be applied as homogeneously as possible because the perforation of the tissue, which is achieved at the end-point of the procedure, may result in the prolapse of liquefied vitreous that will impede further treatment of yet insufficiently ablated parts of the tissue.
The VR simulator renders a highly accurate representation of the various visual indicators that reveal the current ablation state of the tissue. Using the instantaneous restart feature of the simulator, the user can experiment with different laser settings and quickly obtain an intuitive understanding of the coagulation technique. A further benefit of the VR environment is the simulator’s ability to display a heat map of the exposed pars plana tissue either as a HUD or as an overlay element (see Fig. 5).
Potential complications during this procedure originate mostly from poor alignment of the syringe needle with the implant septum. Implementing a sufficiently accurate simulation model was a challenge due to the complex coupling of the relevant scene objects: the implant orientation is coupled to its adjacent sclera tissue and the external forces that drive this interaction result from the penetration of the septum by the syringe needle or its collision with the implant flange. As the needle is flexible, the overall system has a distinct non-linear dynamic. Hence, significant development efforts were required to render the characteristic haptic effects that guide the user during needle insertion.
Early training sessions with the simulator quickly revealed that the users had a tendency to be overconfident about their ability to visually estimate the relative orientation of implant and syringe. Exploiting VR technologies to help the user understand and correct false perceptions became a priority among the training requirements. To this effect, the simulator supports users with abstract guidance elements augmenting the virtual scene. Colored arrows and targeting markers materialize on request and guide the user into the correct position and orientation, see Fig. 2. A replay feature allows the user to observe his own performance from a different perspective and recognize the nature of the cognitive biases causing a previously experienced misperception.
We report on a new and specialized simulation solution in which physicians can train on insertion and refill procedures of an eye implant for intravitreal drug delivery. This solution includes two separate and stand-alone hardware setups with distinct stereoscopic viewing systems, active force feedback input devices with customized exchangeable handpieces and multi-physics interaction models. To the authors’ knowledge, this approach constitutes a novelty and a milestone among the existing simulation solutions in Ophthalmology.
While awaiting the results of the clinical studies that are ongoing, the only indications regarding the construct validity of the simulator training is the positive feedback of the investigators and clinical experts participating in the trial and the associated simulator training sessions ([9, 10]).
To date, Genentech has employed over thirty simulator units for their training efforts.
In consideration of the high financial stakes involved in many clinical trials carried out by pharmaceutical and medical device companies, the significant investment in high-fidelity training devices to support surgical training and thus enable commercialization of new products may be rapidly amortized if there is a measurable improvement of the clinical outcome. Nevertheless, the authors are not aware of any recent trials that were accompanied by similar developments. This publication is at least partially intended to draw attention to this particular use case: the support of clinical trials with VR-based high-fidelity simulators has enormous potential to extend their range of applications to new medical fields and operations for which such a development would otherwise be cost prohibitive.
Availability of data and materials
Prescient & Strategic Intelligence: Section 5.1 - By Offering. Surgical Simulation Market Research Report, LS11131. 2020. p. 55–60.
Campochiaro PA, Marcus DM, Awh CC, Regillo C, Adamis AP, Bantseev V, et al. The port delivery system with ranibizumab for neovascular age-related macular degeneration: results from the randomized phase 2 ladder clinical trial. Ophthalmology. 2019;126(8):1141–54. https://doi.org/10.1016/j.ophtha.2019.03.036.
Panteleimon P, Chorti A, Papagiouvanni I, Paparoidamis G, Drosos C, Panagiotakopoulos T, et al. Virtual and augmented reality in medical education. Intechopen. 2018. https://doi.org/10.5772/intechopen.71963.
Rangarajan K, Davis H, Pucher PH. Systematic review of virtual haptics in surgical simulation: a valid educational tool? J Surg Educ. 2020;77(2):337–47. https://doi.org/10.1016/j.jsurg.2019.09.006.
Bantseev V, Schuetz C, Booler HS, Horvath J, Hovaten K, Erickson S, Bentley E, Nork TM, Freeman WR, Stewart JM, Barteselli G. Evaluation of surgical factors affecting vitreous hemorrhage following Port Delivery System with ranibizumab implant insertion in a minipig model. Retina. 2020;40(8):1520–8. https://doi.org/10.1097/IAE.0000000000002614.
Spraul CW, Grossniklaus HE. Vitreous hemorrhage. Surv Ophthalmol. 1997;42(1):3–39. https://doi.org/10.1016/s0039-6257(97)84041-6.
Lee R, Raison N, Lau WY, Aydin A, Dasgupta P, Ahmed K, Haldar S. A systematic review of simulation-based training tools for technical and non-technical skills in ophthalmology. Eye. 2020;34(10):1737–59. https://doi.org/10.1038/s41433-020-0832-1.
Norman RE, Flanagan JG, Rausch SMK, Sigal IA, Tertinegg I, Eilaghi A, et al. Dimensions of the human sclera: thickness measurement and regional changes with axial length. Exp Eye Res. 2010;90(2):277–84. https://doi.org/10.1016/j.exer.2009.11.001.
Castellanos S. Genentech uses virtual reality to train eye surgeons. The Wall Street Journal, February 6th 2019. https://www.wsj.com/articles/genentech-uses-virtual-reality-to-train-eye-surgeons-11549495028.
Pieramici DJ, Heimann F, Brassard R, Barteselli G, Ranade S. Virtual reality becomes a reality for ophthalmologic surgical clinical trials. Trans Vis Sci Tech. 2020;9–7:1.
The development of the simulators described in this article was funded solely by Genentech Inc.
The custom simulator development described was a collaborative effort of Genentech Inc. and VRmagic GmbH. All authors are employees of either VRmagic GmbH or Genentech Inc.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Heimann, F., Barteselli, G., Brand, A. et al. A custom virtual reality training solution for ophthalmologic surgical clinical trials. Adv Simul 6, 12 (2021). https://doi.org/10.1186/s41077-021-00167-z