Introduction: Pediatric early warning scores (PEWS) identify hospitalized children at risk for deterioration. Manual calculation is prone to human error. Electronic health records (EHRs) enable automated calculation, removing human error. This study's objective was to compare the accuracy of automated EHR-based PEWS calculation (AutoPEWS) to manual calculation and evaluate the non-inferiority of AutoPEWS in predicting deterioration.
Methods: We performed a retrospective cohort study inclusive of non-intensive care unit inpatients at a freestanding children's hospital over 4.5 months in Fall 2018. AutoPEWS mapped the historical manual PEWS scoring rubric to frequently used EHR documentation. We determined accuracy by comparing the expected respiratory subset score based on the current respiratory rate to the actual respiratory score of AutoPEWS and the manual PEWS. The agreement was determined using kappa statistics. We used predicted probabilities from a generalized linear mixed model to calculate areas under the curve for each combination of scores (AutoPEWS, manual) and deterioration outcome (rapid response team activation, unplanned intensive care unit transfer, critical deterioration event). We compared the adjusted difference in areas under the curves between the scores. Non-inferiority was defined as a difference of <0.05.
Results: There were 23,514 total PEWS representative of 5,384 patients. AutoPEWS respiratory scores were 99.97% accurate, while the manual PEWS respiratory scores were 86% accurate. AutoPEWS were higher overall than the manual PEWS (mean 0.65 versus 0.34). They showed a fair-to-good agreement (weighted kappa 0.42). Non-inferiority of AutoPEWS compared with the manual PEWS was demonstrated for all deterioration outcomes.
Conclusions: Automation of PEWS calculation improved accuracy without sacrificing predictive ability.
Copyright © 2020 the Author(s). Published by Wolters Kluwer Health, Inc.