Interpretation of Performance Measures
| Measure | Interpretation | Warning |
|---|---|---|
| Likelihood ratio test (for nested models only) | Significant result: complex model gives better results than simpler model; no significant result: simpler model fits data as well as complex one | Depends on specific context of study and incidence of outcome in population |
| AIC and BIC (for nested and nonnested models) | Lower AIC and BIC indicates better trade-off on model fit and complexity; prognostic index with lowest AIC and BIC is best | |
| Calibration (Brier score) | Brier score closer to 0 means excellent calibration; prognostic index with lowest Brier score is best | |
| Discrimination (AUC and C-index [survival analysis]) | Higher AUC indicates better discriminative ability; higher C-index indicates better discriminative ability | |
| Clinical risk reclassification (categoric and category-free Net Reclassification Index) | Not recommended | |
| Net benefit | Positive net benefit suggests that using prognostic index leads to more appropriate treatment recommendations than treating all patients or treating none | Calculate net benefit according to different weights to proportion of unnecessary intervention in order to evaluate performance and clinical impact across spectrum of decision scenarios |