Table 2
Recalibrating and model revision methods considered for logistic regression models for a prognostic index (PI) with n predictors (ie, methods 1–5) and extension methods considering k new predictors; not included in the original model (ie, methods 6–8)14 30
| No | Updating method | Description | Interpretation | |
| 1 | No adjustment | Fit the original model in exactly the same way it was constructed in the derivation data set. No parameters are estimated. | This method makes no revisions to the model but evaluates the performance of the original scores when applied to an independent cohort. | |
| Recalibration | 2 | α | Correct the ‘calibration in the large’ by updating the intercept α . | If a lack of calibration is noted during the external validation process (ie, 
, 
), an initial ‘re-calibration step’ is performed, in which
α
more or less 
can be updated without changing the PI itself. Updating only the intercept (method 2) addresses problems arising from overly high or low predicted probabilities in the validation data set through only a change in the intercept (
or 
).Beyond updating the intercept, in method 3, the calibration slope can also be updated, in which all original regression coefficients can be multiplied by a constant. This addresses the problem of coefficients in the original model being too large: 
(or too small: 
). |
| 3 |
| Re-estimate both calibration slope 
and intercept
α
. This step is called ‘logistic calibration’. | ||
| Model revision | 4 |
| Building on method 3, each covariate from the original model is included in a forward stepwise procedure using Akaike information criterion (AIC), including only those that lower (improve) the AIC. Their coefficients 
(for i in 1 to n) are then estimated. Of note, their updated regression coefficients 
are then: 
. | In the ‘re-calibration step’ above, the PI is revised globally and equally for all regression coefficients estimated in the derivation data set. In method 4, covariates are included one by one if they improve the relative goodness of the fit of the model (ie, lower the AIC), which identifies covariates having a significantly different effect in the validation versus derivation data sets and then to recalibrate the PI by increasing (
) or decreasing (
) their weights in the model. Notably, in clinical practice, this procedure can account for big changes in covariates over time by recalibrating the model, for example.In method 5, all coefficients are re-estimated in the validation data set. This step also highlights covariates that have a different influence in derivation and validation data sets by comparing α and 
with the original ones. With this method, the original PI is not preserved. |
| 5 |
| Re-estimate coefficients of all the covariates: 
included in the original model and the intercept
α
in the validation data set without any selection. | ||
| Model extension | 6 |
| Re-estimate the intercept
α
, the slope on the PI: 
and the coefficients of the covariates included in the model: 
, as well as coefficients of new covariates: 
in a stepwise manner (as in method 4). | In the ‘model revision’ step, only the original covariates were used. In the ‘model extension’ step, new covariates not originally included in the model are added. Method 6 is similar to method 4 but adds new covariates only if they increase the relative goodness of the fit of the model (ie, lower AIC). For example, in clinical practice, this process enables the incorporation of new covariates into the model should they become available. Method 7 approximates method 5 but, as in method 6, adds new covariates only if they decrease the AIC of the model. Finally, in method 8, we create a completely new model without considering the original one by estimating all the regression coefficients (for covariates originally included as well as new covariates). |
| 7 |
| Fit the exact same model as in the original one; re-estimating all coefficients of the original covariates: 
, without selection process, and adding new covariates 
only if they improve model performance in the validation data set. | ||
| 8 |
| Fit a model which allows the estimation of all the n+k coefficients: 
and the intercept
α
without selection process. |