Establishing Objective Measures of Clinical Competence in Undergraduate Medical Education through Immersive Virtual Reality

Zackoff, Matthew W.; Young, Daniel; Sahay, Rashmi D.; Fei, Lin; Real, Francis J.; Guiot, Amy; Lehmann, Corinne; Klein, Melissa

OBJECTIVE: The Association of American Medical Colleges defines recognition of the need for urgent or emergent escalation of care as a key Entrustable Professional Activity (EPA) for entering residency (EPA#10). This study pilots the use of an immersive virtual reality (VR) platform for defining objective observable behaviors as standards for evaluation of medical student recognition of impending respiratory failure. METHODS: A cross-sectional observational study was conducted from July 2018-December 2019, evaluating student performance during a VR scenario of an infant in impending respiratory failure using the OculusRift™ VR platform. Video recordings were rated by two pair of physician reviewers blinded to student identity. One pair provided a consensus global assessment of performance (not competent, borderline, or competent) while the other used a checklist of observable behaviors to rate performance. Binary discriminant analysis was used to identify the observable behaviors that predicted the global assessment rating. RESULTS: Twenty-six fourth year medical students participated. Student performance of eight observable behaviors was found to be most predictive of a rating of competent, with a 91% probability. Correctly stating that the patient required an escalation of care had the largest contribution towards predicting a rating of competent, followed by commenting on the patient's increased heart rate, low oxygen saturation, increased respiratory rate, and stating that the patient was in respiratory distress. CONCLUSIONS: This study demonstrates that VR can be used to establish objective and observable performance standards for assessment of EPA attainment – a key step in moving towards competency based medical education.