(P176) PREDICTIVE MODELS AID PHYSICIAN PROGNOSTICATION: A SECONDARY ANALYSIS EVALUATING INTEGRATED MODEL AND PHYSICIAN PROGNOSTIC ESTIMATES IN PATIENTS WITH HEART FAILURE WITH REDUCED EJECTION FRACTION

Background: In a recent series of studies from a multicenter Canadian cohort of outpatients with heart failure (HF), we found that model predictions were significantly more accurate than HF cardiologists. In this study, trying to mimic practice, we evaluated the additional predictive value and clinical impact of model predictions to refine physician estimated risk of 1-year mortality by combining model and physician estimates.

METHODS AND RESULTS: We included consented consecutive HF outpatients (LVEF < 40%) followed at 11 HF clinics in Canada. HF cardiologists estimated patient 1-year mortality using their clinical judgment. We calculated model predicted mortality using the Seattle HF Model (SHFM). We followed patients for at least a year to record mortality (or urgent heart transplant or ventricular assist device implant as mortality-equivalent events). Using random forest survival model and cross-validation, we compared the performance SHFM and the HF cardiologist alone, and the integrated HF cardiologist and the SHFM predictions by evaluating model discrimination (c-statistic), calibration (observed vs predicted event rate), risk reclassification and clinical net benefit analyses. Among 1,643 HF patients, 1-year event rate was 9% (95%CI 8%-11%). The SHFM had the adequate discrimination (c-statistic 0.76, 95%CI 0.72-0.80) and excellent calibration while cardiologists showed adequate discrimination (c-statistic 0.75, 95%CI 0.71-0.79 for HF cardiologist) and poor calibration due to significant risk overestimation (Figure 1). When the SHFM estimates were added physician predictions, discrimination significantly improved (0.82, 95%CI 0.78-0.86) with excellent calibration. By risk reclassification analysis, among patients with events, HF cardiologist better reclassified 44% than the SHFM or the integrated model. Among patients without event, however, HF cardiologists worse risk-classified 52% in comparison to SHFM and 71% to the integrated model. By net clinical benefit analysis, when the decision to treat starts at a 1-year mortality < 5%, intervening on all patients or guiding decisions based on physician judgement or the SHFM would result in similar clinical benefit (Figure 2). When the decision to treat involves patient with 1-year mortality of 5% or higher, intervening based on the SHFM predictions would lead to higher benefit than guiding care based on physician judgement. Integrating model and HF cardiologist predictions led to minimally increased benefit in comparison to SHFM alone.

Conclusion: Integrating prediction from the SHFM to physician judgment or using the SHFM alone showed superior accuracy than HF cardiologist predictions, proving that model-informed physician assessment may provide more accurate prognostic information to tailor clinical decision making.