Log in to MyACC
Association of Hospital Participation in a Quality Reporting Program With Surgical Outcomes and Expenditures for Medicare Beneficiaries | Journal Scan
Study Questions:
Does participation in a national quality reporting program make a difference with regard to 30-day mortality, serious complications, reoperation, readmission in 30 days, and costs?
Methods:
The authors used Medicare data and a quasi-experimental study design—that is, a difference-in-differences (or “diff-in-diff”) analysis (see Dimick JB, Ryan AM. JAMA 2014;312:2401 for a primer). In this approach, outcomes in hospitals that participated in the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) were compared with nonparticipating hospitals matched using propensity score methods that included annual surgical volume, baseline risk-adjusted outcomes, and pre-enrollment trends in risk-adjusted outcomes as predictors of participation.
Results:
After accounting for patient and secular factors, there were no statistically significant differences in outcomes at 1, 2, or 3 years after enrollment in ACS NSQIP. Specifically, there were no statistically significant differences in risk-adjusted 30-day mortality (4.3% after enrollment vs. 4.5% before enrollment), serious complications (11.1% after enrollment vs. 11.0% before enrollment), reoperations (0.49% after enrollment vs. 0.45% before enrollment), or readmissions (13.3% after enrollment vs. 12.8% before enrollment). Mean total Medicare payments were also similar, as were payments for the index admission, hospital readmission, and outliers.
Conclusions:
The authors concluded that: “With time, hospitals had progressively better surgical outcomes, but enrollment in a national quality reporting program was not associated with the improved outcomes or lower Medicare payments” in these patients undergoing surgery in ACS NSQIP. Although the collection and feedback of data are critical to quality efforts like ACS NSQIP, this elegant analysis raises the concern that, by itself, such a strategy is likely to be insufficient.
Perspective:
This study is from an experienced surgical outcomes group, and I believe it is a fantastic study. The big take-home (and surprising) message is that little association was found between the national NSQIP QI reporting program and outcomes. This study was elegantly designed, overcoming the biggest Achilles’ heel in contemporary policy evaluation: the lack of adequate control groups. As the authors write, without a control group, most policy evaluations make ‘it difficult to conclude whether improvements in outcomes were truly associated with participation’ or ‘simply represent background trends toward improved outcomes at all hospitals.’ Their use of matching and a ‘diff-in-diff’ analysis is advanced econometrics at its best, and it will be required reading in health economics courses across the country. At the ground level, these findings challenge existing dogma and raise key questions about how reporting efforts in QI may require better designs. They have direct implications for similar registries and programs within cardiology and cardiac surgery. An accompanying JAMA editorial by Don Berwick, the former head of Centers for Medicare and Medicaid Services, is an instant classic.
Clinical Topics: Cardiac Surgery, Invasive Cardiovascular Angiography and Intervention
Keywords: Cardiac Surgical Procedures, Medicare, Propensity Score, Quality Improvement, Reoperation, Risk, Surgeons
< Back to Listings
