Reliability and Validity of Two Surgical Prioritization Systems for Reinstating Non-Emergent Benign Gynecologic Surgery During the COVID-19 Pandemic

Marfori, Cherie Q.; Klebanoff, Jordan S.; Wu, Catherine Z.; Barnes, Whitney A.; Carter-Brooks, Charelle M.; Amdur, Richard L.

STUDY OBJECTIVE: Scientifically evaluate the validity and reproducibility of two novel surgical triaging systems, as well as offer modifications to the MeNTS criteria for improved application in gynecologic surgeries DESIGN: Retrospective cohort study SETTING: Academic university hospital PATIENTS: 97 patients with delayed benign gynecologic procedures due to the COVID-19 pandemic INTERVENTION(S): Surgical prioritization was assessed using two novel scoring systems, the Gyn-MeNTS and mESAS systems for all 93 patients included MEASUREMENTS AND MAIN RESULTS: The inter-rater reliability and validity of two novel surgical prioritization systems (Gyn-MeNTS and mESAS) were assessed. Gyn-MeNTS scores were calculated by three raters and analyzed as continuous variables, with a lower score indicating more urgency/priority. The mESAS score was calculated by two raters and analyzed as a 3-level ordinal variable with a higher score indicating more urgency/priority. All five raters were blinded to reduce bias. Gyn-MeNTS inter-rater reliability was tested using Spearman r and paired t-tests were used to detect systematic differences between raters. Weighted kappa indicated mESAS reliability. Concurrent validity with mESAS and surgeon self-prioritization (SSP) was examined with Spearman r and logistic regression. Spearman r's for all Gyn-MeNTS rater pairs were above 0.80 (0.84 for 1 vs. 2, 0.82 for 1 vs. 3, 0.82 for 2 vs. 3, all p<.0001) indicating strong agreement. The weighted kappa for the 2 mESAS raters was 0.57 (95% CI 0.40-0.73) indicating moderate agreement. When used together, both scores were significantly independently associated with SSP, with strong discrimination (AUC 0.89). CONCLUSIONS: Inter-rater reliability is acceptable for both scoring systems, and concurrent validity of each is moderate for predicting SSP, but discrimination improves to a high level when they are used together.