"How much does having high cholesterol increase my risk of heart attack?" a patient may ask. There is an evidence-based answer, but many clinicians are just as interested in a different number, the predictive value that a patient's elevated low-density lipoprotein (LDL) cholesterol contributes towards accurate 10-year risk assessment. How about the magnitude of effect that treating his or her dyslipidemia will have on preventing a coronary event? That is a separate question entirely.

This intuitive, but easily overlooked, statistical reality is the premise for a meticulously conducted statistical analysis of established risk factors for coronary heart disease (CHD) published recently in Circulation.¹ The authors posit that, when it comes to attributing value to risk factors, there is a need to separate causation from prognostication.

This study at first glance supports well known facts: that hypertension, dyslipidemia, smoking and diabetes, in addition to demographic factors such as age, are associated with cardiovascular risk. That estimating cardiovascular risk is multifactorial and complex. Their major conclusion, though, is that the degree of association of a risk factor or related intervention with cardiovascular event rates is a vastly different metric than the power of that risk factor to predict future events. This finding highlights the importance of proper choice of metric in study design and appropriate interpretation of these metrics in clinical practice.

The authors performed side-by-side comparisons of various metrics for calculating association of classic risk factors with CHD events in a composite primary prevention cohort including Cardiovascular Health Study (CHS), Framingham Offspring Exam, Atherosclerosis Risk In Communities (ARIC), and Multi-Ethnic Study in Atherosclerosis (MESA) study participants. They compared the results of statistical tools designed to provide either prognostication, attributable risk, or estimate of treatment benefit related to five classic CHD risk factors: demographics (combined effect of age, sex, and race), high blood pressure (BP), dyslipidemia (expressed as non-high-density-lipoprotein cholesterol [non-HDL-C]), smoking, and diabetes mellitus (DM). This was published in response to a growing trend of implicating predictive value metrics in the measure of importance of a risk factor.

What they found was intuitive, but not frequently displayed so systematically; that attributing value to cardiovascular risk factors, or to any risk factor for that matter, is heavily dependent on asking the right question. Close association with disease does not automatically indicate strong prognostication.

Conversely, low risk prediction value for pre-morbid risk factors does not automatically translate to low clinical significance, nor should it minimalize the salutary effect of intervention. Of most interest in assessing these risk factors for prognostic power is the authors' variable-added last approach. This displays the quantitative improvement in risk prediction achieved when the risk factor of interest is incorporated into an algorithm with all other risk factors already present.

Discrepancy in Association of Modifiable Risk Factors With CHD

As an example, high cholesterol has well-established causative association with CHD. However, the ability of elevated non-HDL-C to predict 10-year CHD events using the c index (or area under the receiver operating curve) is 0.560; not much better than random chance. For comparison, demographic risk factors alone achieved a c index of 0.685. Further analysis using four models used to report the discriminative or prognostic value of a model or individual risk factor (c-index, NRI, -2 log likelihood, and discrimination slope), suggests a marginal independent effect of non-HDL-C. When the impact of cholesterol level was eliminated from these models, each was still able to achieve between 95% and 98% of the full prognostic value achieved using the model with all risk factors.

However, the hazard ratio tells a different story. Individuals with elevated non-HDL-C are a significant 18% more likely than their counterparts to have an event, even in a model fully adjusted for all other risk factors. The population attributable fraction (PAF) demonstrates that, if dyslipidemia were retroactively erased from the population, an estimated 17% fewer CV events per decade would occur.

It is conceivable, however, that an individual who has never had dyslipidemia is a different substrate than one whose dyslipidemia is treated. Absolute risk reduction (ARR) addresses the latter. Using data-driven estimates for the effect of statin therapy for all with non-HDL-C ≥130 mg/dL, the authors predicted that the average risk of events at the individual level would be reduced by 2.7%. This translates to a 10-year number needed to treat (NNT) of 37 people to prevent one event.

A similar pattern was observed for the other modifiable risk factors. Eliminating any one of these left an algorithm still predictive to ≥89% of the full model, and even the model using demographic factors alone (eliminating all modifiable risk factors) held between 63% and 80% of the prognostic value of the full model.

Alternatively, presence of a single risk factor increased 10-year risk of events between 18% and 107%. Smoking and diabetes were estimated to contribute about 10% each to population disease burden, and high BP had a robust 28% PAF with ARR 3.7% and NNT of 27.

The numerical results in this study must be interpreted with care due to three necessary statistical assumptions, namely that relative risk reduction is preserved across classes of absolute risk, that risk factor profile at baseline exam is preserved throughout follow-up, and the necessary adjustment of baseline BP and non-HDL-C for pre-existing therapy. However, the statistical themes demonstrated above are undisputed and important.

The clinical takeaway from this exercise is that the power of a risk factor to prognosticate is entirely separate from its strength of association to the outcome of interest or from the benefit achieved by intervention.

Therefore, each of the statistical metrics displayed here is trustworthy, but the function of the chosen metric must match the research question. Inappropriate application of these metrics, alternatively, may profoundly overestimate the utility of a new biomarker or mask the clinical effect of a risk factor (as demonstrated with dyslipidemia above).

Appropriate Use of Statistical Metrics to Characterize Risk Factors

Simply put, when the goal is to demonstrate the strength of association of a risk factor with an outcome, hazard ratios may be used. Establishing the population impact of a risk factor or associated intervention is best accomplished using PAF or ARR/NNT, respectively. Finally, to describe the utility of a risk factor or marker in predicting the outcome of interest, one of the model performance metrics (likelihood ratio, c-index, NRI, or discrimination slope) is most useful.

It can be difficult to articulate the clinical meaning of the numerical results of some prognostication metrics; however, hazard ratios, PAF, ARR, and NNT provide numerical outcomes that may be easily grasped by clinicians (Table 1).

TABLE 1: Metrics used in the quantitative analysis of risk factors.^1-3

Metric	Definition	Units / Interpretation	Role	Notes
log likelihood ratio	Assessment of model fit – calculation of the likelihood of outcomes compared to the likelihood of observed outcome predicted by a null model.	Value > 1 indicates model is predictive; higher numbers indicate better fit.	Risk prediction
C statistic; C index	Probability that out of two randomly selected individuals, the model attributes higher risk to one who experiences an event than to one who does not; Probability that out of two randomly selected individuals, the model attributes higher risk to the one who experienced a shorter time-to-event compared to one with longer time-to-event or no event at all.	Value of 1 indicates perfect model prediction and 0.5 is equivalent to random chance. A value of 0.7 generally indicates a good model.	Risk prediction	Based on classification and not absolute risk - low sensitivity for identifying model improvement, particularly when baseline model has good prognostic value. Not influenced by absolute population risk levels.
Net reclassification index (NRI); Reclassification from the null (Maximum relative utility)	Proportion of people who experienced an event who were appropriately attributed higher baseline risk by a new model in comparison to existing model; Variation on the continuous NRI. In comparison to a null model (all are assigned risk equal to population event rate), the proportion of people who experienced an event who were appropriately attributed higher or lower risk via incorporation of new risk factor.	Can be expressed in terms of category (counting only individuals who cross a risk categorization threshold) or continuous (counting all individuals with any upward or downward change in quantitative predicted risk).; Score > 0 indicates independent predictor. NRI ≥0.60 is generally accepted as strong independent predictor.	Risk prediction	Minimally affected by prognostic value of baseline model. Dependent on baseline event rate - assumes weighting of importance based on ratio of events to non-events.; Does not quantify degree of change in risk prediction and therefore may underestimate clinical utility of a model.
Discrimination slope; Integrated discrimination improvement (IDI)	The difference in mean absolute risk predicted in the model between those who experienced an event and those who did not; The difference between discrimination slopes of two models.	Higher values (>0) represent magnitude of absolute risk discrimination (slope) or magnitude of added discrimination (IDI).	Risk prediction	Influenced by baseline event rate
Hazard ratio (HR)	Relative risk of event in those with the risk factor compared with those who do not.	Values >1 indicated elevated risk and 95% confidence interval communicates significance.	Attributable harm of risk factor
Population attributable fraction (PAF)	The expected reduction in absolute risk of event at the population level if the risk factor of interest were not present (optimized).	Absolute percentage	Attributable harm of risk factor	Influenced by prevalence of risk factor in population and event rate.
Absolute risk reduction (ARR)	The expected reduction in mean absolute risk achieved by treatment of risk factor.	Absolute percentage	Attributable benefit of intervention	Based on absolute baseline risk of the population of interest and relative risk reduction achieved with therapy. Better descriptor of clinical significance than relative risk reduction.
Number needed to treat (NNT)	Estimated number of individuals who would need to be treated with the therapy of interest to prevent one event over a pre-specified length of time.	Equal to 1 / ARR.	Attributable benefit of intervention	Better descriptor of clinical significance than relative risk.

Risk Prediction Metrics

Proposal for a new risk prediction model (or addition of a biomarker to an old model) should include both absolute predictive value and improvement from the old model using a model performance metric. What is the difference between the models described in this paper? C-statistic (or C-index, when incorporating time-to-event) are the most commonly accepted metrics for describing the accuracy of a predictive model. However, the relative change in C-index for an individual risk factor is highly influenced by the strength of the baseline model; high baseline value will diminish the individual contribution of that factor and lead to low sensitivity for detecting added discrimination.

In contrast, the improvement in discrimination slope (or the continuous version of this metric, integrated discrimination improvement [IDI]) and NRI are only weakly influenced by the overall model strength.^2,3 Discrimination slopes describe the ability of a model to parse out high risk individuals from low risk individuals, or depict the added power of a new risk factor.

NRI is defined as the incremental value of a novel algorithm or added risk factor to more accurately classify individuals compared to the old model. The binary input of NRI eliminates some of the granular quantification of an individual risk factor – individuals either move either up or down in predicted risk without added input based on degree of change. Additionally, NRI and discrimination slope are influenced by baseline event rate and model calibration, while c-index is not.^2-4

Risk Attribution Metrics

Hazard ratio, the metric likely most familiar to readers, is used to establish the magnitude of association of a risk factor to the outcome of interest at the individual level. PAF, ARR, and NNT, in contrast, are population metrics, based on the actual prevalence of the risk factor in the population of interest and the baseline event rate in that population. PAF is the degree to which the presence of a risk factor contributes to the event rate in a population. PAF is highly dependent on the chosen threshold used to define high risk. ARR and/or NNT most reliably communicate the observed or potential impact of intervention targeting the risk factor of interest. Therefore, these values are highly valuable in population preventive health; however, it is important to avoid the potential pitfall of assuming the findings in one population are generalizable to another.

PAF may be interpreted as the estimated effect of primordial prevention on that risk factor, while ARR represents the potential benefit of primary prevention in the presence of the risk factor. Even the relative trends for PAF and ARR across subgroups are not always linked. The authors demonstrated this using separation of the population by age – with increasing age, the relative PAF of each modifiable risk factor was diminished, simply due to the rising proportional effect of age as a risk factor. However, ARR for anti-hypertensive or lipid-lowering therapy was higher in older age groups, attributable to the higher absolute risk in this demographic.

Clinical Application: Improving Risk Assessment

There is an ongoing quest for improved calibration and discrimination in ASCVD risk prediction algorithms. This study serves as a reminder that causative risk factors do not automatically offer accurate quantitative risk prediction. This lends support to the ongoing investigation into alternative prognostic markers, particularly biomarkers of existing subclinical atherosclerosis which may improve detection of high risk individuals, particularly in the absence of traditional risk factors.^5-10 For example, individuals with no traditional risk factors and very high coronary artery calcium score (CAC) were found to have a substantially higher mortality rate compared to individuals with ≥3 risk factors but absent CAC.¹¹ There is also room to incorporate cumulative exposure to risk factors into risk prediction algorithms.^12,13

In an attempt the achieve the most accurate risk prediction, there should be appropriate emphasis on maximizing discrimination at clinically important risk cut-offs, which may in theory be distinct from those with the highest c-statistic across risk levels.

Alternatively, some have suggested that in allocation of risk-modifying therapy, estimated ARR (the potential impact of therapy itself) should be considered in addition to absolute estimated risk. In fact, when statin allocation was modeled in the NHANES population using a threshold of estimated absolute risk reduction, rather than absolute risk, the statin eligible population increased by a factor of 1.6 (primarily adding younger patients with higher LDL-C), with a NNT of 25 in the expanded group compared with NNT 21 in the group deemed statin-eligible under current guidelines.¹⁴

Clinical Application: Preventive Care

Finally, the discrepancy between PAF and estimated ARR, as well as the finding that the highest PAF for modifiable risk factors lies in younger age groups, adds further weight to the mountain of evidence promoting primordial prevention as an optimal population health strategy to reduce the global burden of CHD. As stated by the authors, "We should shift away from telling patients how much their habits increase risk of ASCVD event and towards sharing the news that taking active steps to improve modifiable risk factors can…prevent or delay catastrophic events."¹

To put it another way: imagine your house is in the direct path of a storm's projected course. That fact alone is the strongest predictor of storm-related damage to your house. However, all who find themselves in this scenario wound make sure to address any modifiable risk factors which may delay or diminish damage – boarding up windows, securing broken shingles, etc. In the same way, we have adequate tools for identifying patients at high risk of cardiac event. Ongoing improvement in statistical evaluation of causative risk factors may or may not improve our ability to categorize these individuals. However, the data presented here serve as a reminder that early intervention to address these modifiable factors can have a powerful impact in preventing or delaying CHD events.

References

1. Pencina MJ, Navar AM, Wojdyla D, et al. Quantifying importance of major risk factors for coronary heart disease. Circulation 2018. [Epub ahead of print]
2. Pencina MJ, D'Agostino RB, Vasan RS. Stastical methods for assessment of added usefulness of new biomarkers. Clin Chem Lab Med 2010;48:1703-11.
3. Cook NR. Assessing the incremental role of novel and emerging risk factors. Curr Cardiovasc Risk Rep 2010;4:112-9.
4. Pencina MJ, D'Agostino RB, Pencina KM, Janssens AC, Greenland P. Interpreting incremental value of markers added to risk prediction models. Am J Epidemiol 2012;176:473-81.
5. Akintoye E, Briasoulis A, Afonso L. Biochemical risk markers and 10-year incidence of atherosclerotic cardiovascular disease: independent predictors, improvement in pooled cohort equation, and risk reclassification. Am Heart J 2017;193:95-103.
6. McClelland RL, Jorgensen NW, Budoff M, et al. 10-year coronary heart disease risk prediction using coronary artery calcium and traditional risk factors: derivation in the MESA (Multi-Ethnic Study of Atherosclerosis) with validation in the HNR (Heinz Nixdorf Recall) study and the DHS (Dallas Heart Study). J Am Coll Cardiol 2015;66:1643-53.
7. Wang TJ, Gona P, Larson MG, et al. Multiple biomarkers for the prediction of first major cardiovascular events and death. N Engl J Med 2006;355:2631-9.
8. Grundy SM, Stone NJ, Bailey AL, et al. 2018 AHA/ACC/AACVPR/AAPA/ABC/ACPM/ADA/AGS/APhA/ASPC/NLA/PCNA guideline on the management of blood cholesterol: a report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines. J Am Coll Cardiol 2018. [Epub ahead of print]
9. Polonsky TS, McClelland RL, Jorgensen NW, et al. Coronary artery calcium score and risk classification for coronary heart disease prediction. JAMA 2010;303:1610-6.
10. Martin SS, Blaha MJ, Blankstein R, et al. Dyslipidemia, coronary artery calcium, and incident atherosclerotic cardiovascular disease: implications for statin therapy from the multi-ethnic study of atherosclerosis. Circulation 2014;129:77-86.
11. Nasir K, Rubin J, Blaha MJ, et al. Interplay of coronary artery calcification and traditional risk factors for the prediction of all-cause mortality in asymptomatic individuals. Circ Cardiovasc Imaging 2012;5:467-73.
12. Nvar-Boggan AM, Peterson ED, D'Agostino RB Sr., Neely B, Sniderman AD, Pencina MJ. Hyperlipidemia in early adulthood increases long-term risk of coronary heart disease. Circulation 2015;131:451-8.
13. Allen NB, Siddigue J, Wilkins JT, et al. Blood pressure trajectories in early adulthood and subclinical atherosclerosis in middle age. JAMA 2014;311:490-7.
14. Thanassoulis G, Williams K, Altobelli KK, Pencina MJ, Cannon CP, Sniderman AD. Individualized statin benefit for determining statin eligibility in the primary prevention of cardiovascular disease. Circulation 2016;133:1574-81.

Keywords: Antihypertensive Agents, Calibration, Calcium, Atherosclerosis, Algorithms, Cardiovascular Diseases, Coronary Vessels, Coronary Disease, Demography, Cholesterol, LDL, Cholesterol, Diabetes Mellitus, Biomarkers, Pharmacological, Dyslipidemias, Early Intervention, Educational, Follow-Up Studies, Herpes Zoster, Hydroxymethylglutaryl-CoA Reductase Inhibitors, Lipids, Hypertension, Lipoproteins, Myocardial Infarction, Odds Ratio, Primary Prevention, Risk Assessment, Risk Factors, Smoking, Numbers Needed To Treat

< Back to Listings