Article Text
Abstract
Background The QCovid algorithm is a risk prediction tool that can be used to stratify individuals by risk of COVID-19 hospitalisation and mortality. Version 1 of the algorithm was trained using data covering 10.5 million patients in England in the period 24 January 2020 to 30 April 2020. We carried out an external validation of version 1 of the QCovid algorithm in Scotland.
Methods We established a national COVID-19 data platform using individual level data for the population of Scotland (5.4 million residents). Primary care data were linked to reverse-transcription PCR (RT-PCR) virology testing, hospitalisation and mortality data. We assessed the performance of the QCovid algorithm in predicting COVID-19 hospitalisations and deaths in our dataset for two time periods matching the original study: 1 March 2020 to 30 April 2020, and 1 May 2020 to 30 June 2020.
Results Our dataset comprised 5 384 819 individuals, representing 99% of the estimated population (5 463 300) resident in Scotland in 2020. The algorithm showed good calibration in the first period, but systematic overestimation of risk in the second period, prior to temporal recalibration. Harrell’s C for deaths in females and males in the first period was 0.95 (95% CI 0.94 to 0.95) and 0.93 (95% CI 0.92 to 0.93), respectively. Harrell’s C for hospitalisations in females and males in the first period was 0.81 (95% CI 0.80 to 0.82) and 0.82 (95% CI 0.81 to 0.82), respectively.
Conclusions Version 1 of the QCovid algorithm showed high levels of discrimination in predicting the risk of COVID-19 hospitalisations and deaths in adults resident in Scotland for the original two time periods studied, but is likely to need ongoing recalibration prospectively.
- COVID-19
- clinical epidemiology
Data availability statement
All code, metadata and documentation for this project is publicly available at https://github.com/EAVE-II/Qcovid-validation. A data dictionary is available at https://github.com/EAVE-II/EAVE-II-data-dictionary. Most of the data that were used in this study are highly sensitive and will not be made available publicly.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Data availability statement
All code, metadata and documentation for this project is publicly available at https://github.com/EAVE-II/Qcovid-validation. A data dictionary is available at https://github.com/EAVE-II/EAVE-II-data-dictionary. Most of the data that were used in this study are highly sensitive and will not be made available publicly.
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @DrAzizSheikh
Contributors AS and JH-C conceptualised the study. CR carried out the formal analysis. CRS wrote the initial draft of the manuscript. SK and EM assisted with the statistical analysis. SK wrote later versions of the manuscript. All authors assisted with review and editing. CR, EM and EV have verified the underlying data. CR is the guarantor for this work.
Funding Medical Research Council (MR/R008345/1), National Institute for Health Research Health Technology Assessment Programme, funded through the UK Research and Innovation Industrial Strategy Challenge Fund Health Data Research UK.
Competing interests JH-C reports grants from MRC, grants from Wellcome Trust, grants from NIHR, during the conduct of the study; other from ClinRisk, outside the submitted work. AS reports grants from NIHR, grants from MRC, grants from HDR UK, during the conduct of the study.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.