Predictive accuracy of computer-aided versions of the on-admission National Early Warning Score in estimating the risk of COVID-19 for unplanned admission to hospital: a retrospective development and validation study

Faisal, M.; Mohammed, M. A.; Richardson, D.; Steyerberg, E. W.; Fiori, M.; Beatson, K.

Objectives: To consider the potential of the National Early Warning Score (NEWS2) for COVID-19 risk prediction on unplanned admission to hospital. Design: Logistic regression model development and validation study using a cohort of unplanned emergency medical admission to hospital. Setting: York Hospital (YH) as model development dataset and Scarborough Hospital (SH) as model validation dataset. Participants: Unplanned adult medical admissions discharged over 3 months (11 March 2020 to 13 June 2020 ) from two hospitals (YH for model development; SH for external model validation) based on admission NEWS2 electronically recorded within 24 hours of admission. We used logistic regression modelling to predict the risk of COVID-19 using NEWS2 (Model M0') versus enhanced cNEWS models which included age + sex (model M1') + subcomponents (including diastolic blood pressure + oxygen flow rate + oxygen scale) of NEWS2 (model M2'). The ICD-10 code U071 was used to identify COVID-19 admissions. Model performance was evaluated according to discrimination (c statistic), calibration (graphically), and clinical usefulness at NEWS2 [≥]5. Results The prevalence of COVID-19 was higher in SH (11.0%=277/2520) than YH (8.7%=343/3924) with higher index NEWS2 (3.2 vs 2.8) but similar in-hospital mortality (8.4% vs 8.2%). The c-statistics for predicting COVID-19 for cNEWS models (M1',M2') was substantially better than NEWS2 alone (M0') in development (M2': 0.78 (95%CI 0.75-0.80) vs M0' 0.71 (95%CI 0.68-0.74)) and validation datasets (M2': 0.72 (95%CI 0.69-0.75) vs M0' 0.65 (95%CI 0.61-0.68)). Model M2' had better calibration than Model M0' with improved sensitivity (M2': 57% (95%CI 51%-63%) vs M0' 44% (95%CI 38%-50%)) and similar specificity (M2': 76% (95%CI 74%-78%) vs M0' 75% (95%CI 73%-77%)) for validation dataset at NEWS2[≥]5. Conclusions Model M2' is reasonably accurate for predicting the on-admission risk of COVID-19. It may be clinically useful for an early warning system at the time of admission especially to triage large numbers of unplanned hospital admissions.