Article Text

Original research
External validation of the QCovid risk prediction algorithm for risk of COVID-19 hospitalisation and mortality in adults: national validation cohort study in Scotland
  1. Colin R Simpson1,2,
  2. Chris Robertson3,
  3. Steven Kerr2,
  4. Ting Shi2,
  5. Eleftheria Vasileiou2,
  6. Emily Moore4,
  7. Colin McCowan5,
  8. Utkarsh Agrawal5,
  9. Annemarie Docherty2,
  10. Rachel Mulholland2,
  11. Josie Murray6,
  12. Lewis Duthie Ritchie7,
  13. Jim McMenamin6,
  14. Julia Hippisley-Cox8,
  15. Aziz Sheikh2
  1. 1 School of Health, Victoria University of Wellington, Wellington, New Zealand
  2. 2 Usher Institute, The University of Edinburgh, Edinburgh, UK
  3. 3 Department of Mathematics and Statistics, University of Strathclyde, Glasgow, UK
  4. 4 Information Services Division, Public Health Scotland, Edinburgh, UK
  5. 5 School of Medicine, University of St Andrews, St Andrews, UK
  6. 6 Health Protection Scotland, Public Health Scotland, Glasgow, UK
  7. 7 Academic Primary Care, University of Aberdeen, Aberdeen, UK
  8. 8 Primary Care Health Sciences, University of Oxford, Oxford, UK
  1. Correspondence to Professor Colin R Simpson, School of Health, Victoria University of Wellington, Wellington, New Zealand; colin.simpson{at}vuw.ac.nz

Abstract

Background The QCovid algorithm is a risk prediction tool that can be used to stratify individuals by risk of COVID-19 hospitalisation and mortality. Version 1 of the algorithm was trained using data covering 10.5 million patients in England in the period 24 January 2020 to 30 April 2020. We carried out an external validation of version 1 of the QCovid algorithm in Scotland.

Methods We established a national COVID-19 data platform using individual level data for the population of Scotland (5.4 million residents). Primary care data were linked to reverse-transcription PCR (RT-PCR) virology testing, hospitalisation and mortality data. We assessed the performance of the QCovid algorithm in predicting COVID-19 hospitalisations and deaths in our dataset for two time periods matching the original study: 1 March 2020 to 30 April 2020, and 1 May 2020 to 30 June 2020.

Results Our dataset comprised 5 384 819 individuals, representing 99% of the estimated population (5 463 300) resident in Scotland in 2020. The algorithm showed good calibration in the first period, but systematic overestimation of risk in the second period, prior to temporal recalibration. Harrell’s C for deaths in females and males in the first period was 0.95 (95% CI 0.94 to 0.95) and 0.93 (95% CI 0.92 to 0.93), respectively. Harrell’s C for hospitalisations in females and males in the first period was 0.81 (95% CI 0.80 to 0.82) and 0.82 (95% CI 0.81 to 0.82), respectively.

Conclusions Version 1 of the QCovid algorithm showed high levels of discrimination in predicting the risk of COVID-19 hospitalisations and deaths in adults resident in Scotland for the original two time periods studied, but is likely to need ongoing recalibration prospectively.

  • COVID-19
  • clinical epidemiology

Data availability statement

All code, metadata and documentation for this project is publicly available at https://github.com/EAVE-II/Qcovid-validation. A data dictionary is available at https://github.com/EAVE-II/EAVE-II-data-dictionary. Most of the data that were used in this study are highly sensitive and will not be made available publicly.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Data availability statement

All code, metadata and documentation for this project is publicly available at https://github.com/EAVE-II/Qcovid-validation. A data dictionary is available at https://github.com/EAVE-II/EAVE-II-data-dictionary. Most of the data that were used in this study are highly sensitive and will not be made available publicly.

View Full Text

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • Twitter @DrAzizSheikh

  • Contributors AS and JH-C conceptualised the study. CR carried out the formal analysis. CRS wrote the initial draft of the manuscript. SK and EM assisted with the statistical analysis. SK wrote later versions of the manuscript. All authors assisted with review and editing. CR, EM and EV have verified the underlying data. CR is the guarantor for this work.

  • Funding Medical Research Council (MR/R008345/1), National Institute for Health Research Health Technology Assessment Programme, funded through the UK Research and Innovation Industrial Strategy Challenge Fund Health Data Research UK.

  • Competing interests JH-C reports grants from MRC, grants from Wellcome Trust, grants from NIHR, during the conduct of the study; other from ClinRisk, outside the submitted work. AS reports grants from NIHR, grants from MRC, grants from HDR UK, during the conduct of the study.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Linked Articles