National Cardiovascular Data Registry (NCDR) hospitals participating in any of six hospital-based clinical registry programs receive feedback on their quarterly performance in institutional outcomes reports. Executive summary metrics highlight key processes of care (e.g. medication use or PCI within 90 minutes) and outcomes (e.g. mortality, adverse events). These metrics are primarily based on current ACC/AHA clinical practice guidelines and/or performance measures. Hospitals receive their performance for the previous four quarters compared with aggregate benchmarks of all registry participants to guide internal quality improvement efforts.

An important concern regarding raw outcome metrics, particularly those used for the purposes of accountability, is that they do not account for important differences in hospital case mix. In 1998, a risk-adjustment model for in-hospital mortality was developed using the detailed clinical data within the CathPCI Registry^®. Risk-adjustment models require significant additional resources and technical expertise for development, validation, and implementation compared with reporting unadjusted outcome rates. Nevertheless, NCDR leadership felt that risk adjusted outcomes metrics were necessary to provide fair estimates of rates of outcomes in order to better understand the quality and safety of care at an institution.

Risk-adjustment models are designed to estimate patients' risk of adverse outcomes given the array of pre-procedural risk factors related to the outcome so that an institution's "expected" outcome rate can be calculated. Models are developed by identifying variables that are influence the outcome. Factors related to the procedure (such as vascular access site) or physician decisions (e.g. medication use) are often excluded as they can be modified in the care of the patient and may reflect the skill and quality of decision making by the physician rather than the inherent risk level of the patient. Statistical analysis is used to determine the set of variables and associated coefficients based on the extent to which variables impact the outcomes.
Hospital expected rates are calculated based on logistic regression analysis to estimate the probability of experiencing the outcome for each patient given that patient’s risk factors. Each hospital's observed or actual outcome rate is divided by the expected rate to obtain an "observed to expected ratio." A ratio of one indicates that the hospital actual outcome rate is exactly as predicted by the risk model. Rates over one (observed rates higher than expected) are seen when a hospital’s rate is higher than the risk model predicted based on the case mix of that hospital. These ratios can be multiplied by the registry aggregate average to display a "risk-adjusted rate."

In order to ensure the validity of risk models, in addition to statistical validation the NCDR has data quality processes in place to ensure data used to calculate hospital risk adjusted rates are complete and accurate. Once a model is established, data quality thresholds for required completeness for data submissions are established to minimize the presence of missing variables.  Typically, data quality thresholds for variables in a risk model are set to be at least 90 percent.  In addition, a minimal number of imputation rules are set for missing data. For example, if a height or weight was missing to compute body mass index, the patient's BMI was imputed to a sex-specific median.

When the risk adjusted measures are reported in the NCDR institutional outcomes reports, each individual hospital's observed rate is divided by the hospital's expected rate, multiplied by the observed rate for all NCDR participating hospitals, and then reported as a rate. Risk model reporting requires that at least one event occurred during the reporting period in order to report a value. Thus, if there were no events during that period, the unadjusted observed result would be reported as a zero, but the risk adjusted rate would be reported as a null or no value appearing in the report. While rare, this situation does occasionally occur, usually for very low volume hospitals.

While risk models do not adjust for hospital volume, the NCDR is evolving the risk models reported in the institutional outcomes reports to adopt a hierarchical approach. As described above, the traditional models account for patient characteristics. In essence, hierarchical models also account for hospital characteristics. For example, a traditional model calculates a patient's risk of mortality based on risk factors for the entire registry population. Hierarchical models assess patients' risk of mortality based on his or her demographics and comorbidities within the hospital where care is being provided. Therefore, hierarchical models calculate the odds of experiencing an outcome for a similar or "like" patient at a particular hospital. Taking into account hospital characteristics that might affect the degree to which factors predict a patient's risk of experiencing a particular outcome, hierarchical models tend to more accurately predict risk. The following diagrams further illustrate the differences in approaches.

Non-Hierarchical Modeling

Hierarchical Modeling

Regardless of risk modeling approach, one of three scenarios are possible that either the hospital’s risk adjusted rate is higher, lower, or about the same as the overall registry. Obviously, the most concern is generated when a hospital has a risk-adjusted rate that is substantially higher than the overall registry rate. When reviewing the data, hospitals should begin with a systematic analysis of the data that was submitted to the NCDR. Data accuracy  should be assured, and chance variation related to sampling may affect these models. However, the primary goal of the feedback is to stimulate local quality improvement: evaluation by hospitals of their care processes with the goal of identifying targets for improvement. The NCDR "community" is also a resource for best practices to optimize care delivery and patient outcomes. It is also recommended that hospitals observe their results over time to compare trends and make determinations regarding potential issues with care being provided.

In addition to PCI mortality, risk-adjustment models have been developed or are being developed in the following areas:

Registry	Model	Status
CathPCI Registry	Bleeding (PCI Patients)	In outcomes reports
CathPCI Registry	Acute Kidney Injury (PCI Patients)	In outcomes reports, late 2012
Action Registry®-GWTGTM	Bleeding	In outcomes reports
Action Registry®-GWTGTM	Mortality	In outcomes reports
CARE Registry	Outcomes TBD	Development underway
ICD Registry	Complications	Development underway
Impact	Outcomes TBD	Proposed for 2013

In sum, national clinical registry programs like the NCDR can employ detailed clinical data in large patient populations to develop robust risk models that provide more information to hospitals about their quality of care. Physicians are encouraged to work with their hospital's Registry Site Manager to better understand the risk models, as well as all metrics, in the quarterly institutional reports. If you are familiar with your hospital's RSM, please contact the NCDR by e-mail at ncdr@acc.org, with your name, hospital name, hospital address, phone number and a brief description of your inquiry. You also can contact the NCDR™ Customer Support line at 800-257-4737.

 

 

< Back to Listings