Log in to MyACC
Can a Teaching Intervention Reduce Interobserver Variability in LVEF Assessment: A Quality Control Exercise in the Echocardiography Lab
Study Questions:
Can a formal quality assurance teaching intervention (QATI) reduce interobserver variability (IOV) for the visual estimation of left ventricular ejection fraction (LVEF) within a group of sonographers and physicians?
Methods:
LVEF was estimated by 25 participants including 14 sonographers and 11 physicians with 1-35 years of experience. Complete echocardiograms from 14 cases with a range of LVEF and image quality were viewed by the group of 25 participants, who then provided a visual estimate of LVEF. LVEF was also determined by one observer using biplane Simpson’s rule. “Disagreement” was defined as an absolute difference of visually estimated LVEF from quantitative EF of more than ± 0.05. The 14 originally evaluated cases were represented, along with graphic plots of LVEF for each participant compared to Simpson’s rule LVEF. Three 1-hour sessions were held reviewing the individual data compared to Simpson’s rule LVEF with a facilitated discussion of reasons for error, method for identifying LV endocardium, and review of 10 different reference cases with discussion of the effect of wall motion abnormalities, identification of LV endocardium, and effect of arrhythmia. The biplane Simpson’s LVEF was displayed and reviewed for each case for comparison. Three months following the initial assessment, 14 new cases were presented to the group for analysis. Finally, a subgroup of eight participants underwent repeat evaluation 12 months following the teaching intervention by reviewing 14 additional cases.
Results:
For the preintervention echocardiograms, IOV for visual estimation of LVEF was ± 0.14 EF. IOV significantly declined to ± 0.08 EF following the teaching intervention (F = 2.8, p = 0.007). This represents a 40% reduction in IOV. Notably, evaluators with <5 years and >5 years had no difference in degree of improvement. Preintervention: 44% of cases were misclassified (>0.5 variance). The misclassification rate was 66% for cases with EF 0.30-0.55, and 17% for cases with EF outside of that range. Post-intervention, the misclassification rate dropped to 21%.
Conclusions:
The IOV for visual estimation of LVEF can be dramatically improved by a targeted teaching intervention, which results in a 40% reduction in IOV for observers of a broad range of experience.
Perspective:
The visual estimation of LVEF is a necessary element for echocardiographic reporting. It is commonly employed in virtually all laboratories, and prior studies have demonstrated relatively good correlation with quantitative LVEF in experienced hands. In addition to providing a rapid semi-quantitative assessment of LV function, the ability to visually estimate LVEF is a requisite component of quantitative determination of LVEF, as it serves as an internal check to the validity of endocardial border detection and outlining. Discrepancies among observers and misclassification of LVEF are common with visual estimation alone, but have not previously been the target of a formalized teaching program for improving LVEF assessment as was done here. This study involved a series of six 1-hour sessions within the construct of an echocardiography conference with structured discussion of individual cases with respect to reasons for error, and structured formalized teaching for avoiding error and improving accuracy. The intervention resulted in substantial improvement in IOV and a marked reduction in the rates of misclassification of echocardiograms. As routine ongoing quality assurance programs are mandated for laboratory accreditation, and accurate determination of LVEF probably remains problematic in most laboratories, the model developed here would appear to be an excellent guide for other laboratories.
Keywords: Physicians, Laboratories, Endocardium, Observer Variation
< Back to Listings
