The ACC/American Heart Association (AHA) risk score for atherosclerotic cardiovascular disease (ASCVD) “shows an overestimation of ASCVD risk in a modern, multiethnic cohort without baseline clinical ASCVD,” according to a study published Feb. 16 in Annals of Internal Medicine.

The study compared five risk scores – the ACC/AHA ASCVD risk score, the Reynolds Risk Score, and three based on the Framingham risk score (FRS): FRS-CHD, FRS-CVD and ATPIII-FRS-CHD – in 4,227 patients aged 50 to 74 years without diabetes at baseline enrolled in the Multi-Ethnic Study of Atherosclerosis (MESA), a community-based, sex-balanced, multiethnic cohort.

Results showed that the new ACC/AHA ASCVD and the three Framingham-based risk scores, “overestimated cardiovascular events by 37 percent to 154 percent in men and 8 percent to 67 percent in women. Overestimation was noted throughout the continuum of risk. In contrast, the Reynolds Risk Score overestimated risk by 9 percent in men but underestimated risk by 21 percent in women.”

The authors note that “medication use and interim revascularizations are not robust explanations of the overestimation,” and conclude that “physicians treating patients similar to those in MESA may consider interpreting the absolute risk generated by the new risk score with caution.”

According to Donald M. Lloyd-Jones, MD, FACC, and David Goff, Jr., MD, PhD, co-chairs of the ACC/AHA ASCVD Risk Assessment Work Group, “the general findings of this analysis are not news. The guideline panel noted that there are few contemporary natural history cohorts that allow for appropriate validation of the ACC/AHA equations. However, the guideline panel already explored and reported the calibration of the ACC/AHA equations in MESA in the guideline document and noted some over-estimation of risk. The guideline panel posited several possible explanations for the over-estimation, which the authors of the present paper have attempted to explore in more detail. However, there are significant concerns with their methodological approach which likely biased their findings:

1. The authors included all MESA participants, including Chinese-Americans and Hispanic-Americans, which is clearly contributing to over-estimation. The equations were derived in and strongly recommended for use in Non-Hispanic Whites and African-Americans, with a weak recommendation for use in Chinese- and Hispanic-Americans, who represent more than one-third of the participants in the current report. The guideline noted that the equations would overestimate risk in these subgroups and called for further research to improve risk assessment in these groups. A more appropriate analysis would have would have looked at each ethnic group separately in order to provide insight on this important issue.
2. MESA, although contemporary, may not be an appropriate cohort for validation of an equation intended to predict the natural history of ASCVD, given that all participants underwent coronary artery calcium (CAC) scoring at least twice. We know from other MESA publications that participant knowledge of this information was associated with greater downstream initiation of preventive therapies, especially among those at higher risk or with higher CAC scores. Thus, those that were more likely to have ASCVD events were also more likely to be treated with preventive therapies, so those events would be avoided. This was undoubtedly a major contributor to the perceived over-estimation by the equations.
3. The risk equations were developed to predict the ASCVD events that would occur over 10 years if people were never treated. The authors’ data show that well over 80 percent of the MESA participants were treated with one or more preventive medications (including aspirin, statins, antihypertensive therapy or elective revascularization) during follow up. Furthermore, only 6.4 percent of the observed ASCVD events occurred in untreated individuals. It is great news from a public health perspective that MESA participants were so well treated for ASCVD prevention, but this high level of treatment renders this cohort inappropriate for evaluating how the equations work in an untreated natural history cohort, the target of the equations. MESA may be a very valuable cohort for understanding residual risk in a highly treated population.
4. In the present paper the authors attempted to account for the effects of downstream preventive therapy with several methods, all of which appear to be flawed. The authors used simple statistical adjustment for initiation of downstream therapy, which does not account for the effects of that therapy in reducing risk. Second, they censored individuals at the time of starting lipid therapy, which does not account for the events prevented by that therapy. Finally, they excluded all participants who were ever treated with preventive therapies, yielding a highly biased analysis based on only 14 ASCVD events – clearly an unstable estimate.
5. The relative ratio of congestive heart failure and angina events to coronary heart disease (CHD) or stroke events is much greater than might be expected in a natural history cohort, suggesting that the preventive interventions provided to those at higher risk for CHD and stroke may well have altered their natural history to other types of outcomes not predicted by the ACC/AHA equations.

“Taken together, these concerns raise important questions about the appropriate interpretation of the present findings and suggest the need for improved methods for such analyses.  Furthermore and more importantly for our shared goal of prevention of ASCVD, these results underscore the challenges facing future attempts to update risk equations for ASCVD.  The wonderful progress made over the past several decades in developing and implementing preventive therapies for ASCVD has contributed greatly to the reductions in ASCVD mortality observed during the same period.  This same progress has made identification of natural history cohorts impossible.  Future updates of risk equations will require new and valid methods for accounting for the impact of preventive interventions.  Alternatively, new paradigms for selecting individuals for preventive interventions may be needed.  As MESA has shown us, screening for subclinical atherosclerosis shows great promise as such an approach.”

“The guidelines, while not perfect, represent our best current evidence-based approach to furthering the mission of preventing ASCVD. The paradigm of matching the intensity of preventive therapy to the level of ASCVD risk is a foundation of this approach. The ACC/AHA Pooled Cohort Equations represent the best current approach to ASCVD risk prediction for non-Hispanic Whites and African Americans. These equations represent a major step forward by addressing risk in African Americans for the first time as well as by incorporating risk for stroke. We welcome efforts to improve risk assessment as well as efforts to examine new paradigms for guiding preventive interventions, such as screening for subclinical disease. While awaiting these new scientific advances, we recall the advice to not let the perfect be the enemy of the good. Implementation of these guidelines will result in additional public health benefit through further reductions in ASCVD. Let’s get on with that effort while pursuing new knowledge to improve future guidelines.”

Keywords: American Heart Association, Antihypertensive Agents, Arteriosclerosis, Aspirin, Atherosclerosis, Calcium, Calibration, Coronary Artery Disease, Diabetes Mellitus, European Continental Ancestry Group, Heart Failure, Lipids, Public Health, Risk Assessment, Stroke

< Back to Listings