More Studies Assessing Quality Improvement Using Electronic Health Records

Earlier this year, the informatics world was abuzz with a study published in Archives of Internal Medicine by Romano and Stafford that found a lack of improvement in healthcare quality measures for patients whose physicians had adopted electronic health records (EHRs) [1]. As I detailed in a posting to this blog, as well as in a co-authored letter to the editor that was published in Archives [2], this study had a number of flaws. My main complaint with the study was that the quality measures assessed were independent of the EHR intervention, hence any association, positive or negative, was indirect at best.

The furor about the paper died down, and most people got back to working on implementing meaningful use. No one disagreed that we need more research on whether EHR systems do improve healthcare quality, including studies with better methodology.

Last month, another study came along. Published in the New England Journal of Medicine (NEJM) by Cebul et al., this study used a somewhat similar methodology to assess 46 practices in the Cleveland area, 33 of which had adopted EHRs [3]. The study assessed the outcomes of 27,207 patients with diabetes mellitus who were followed by a total of 569 providers. The study looked at four process measures and five outcome measures in those diabetic patients, comparing them for providers who had and had not adopted EHRs. Overall composite quality measures were developed for the process and outcome measures, and found to be 35.1% higher in the former and 15.2% higher in the latter. The difference was found to persist across all insurance types and, even more gratifying, for "safety net" clinics that historically see more complicated patients of lower socioeconomic status.

This study did use a roughly similar methodology to the Romano and Stafford study, and as such must be viewed as having a weaker form of evidence than a direct randomized controlled trial (RCT). Of course, in reality, such an RCT would be near impossible to do, i.e., randomizing patients to receive their care from a provider having an EHR or not. We also know that there can be confounders between practices utilizing and not utilizing EHRs.

Nonetheless, this study did have advantages over similar studies done before it, including the Romano and Stafford study. One clear advantage was that the study had complete data on all patients (unlike the Romano and Stafford study that only relied on a data set from the CDC National Center for Health Statistics (NCHS). The researchers also had precise data on the providers, the EHR implementation, and how the quality measures were integrated into the provision of care.

While this new study received a great deal of press, another study that received less press, which was published shortly after the publication of the Romano and Stafford study, should have received more [4]. Although still not an RCT design,  this study did use a before-and-after methodology to examine change in compliance with 16 quality measures before and after implementation of a commercial EHR in a large academic internal medicine practice. The results showed improvement after the EHR was implemented.

In an editorial accompanying the Cebul et al. study, Classen and Bates noted that the new NEJM study showed the "meaning in meaningful use" [5]. They correctly point out that implementing EHRs is not what HITECH should be about, but rather showing that the technology can be used to make meaningful improvement in the health of patients whose providers use it. As in most areas of medicine, we cannot wait for the perfect study or studies to answer all questions unequivocally, but the evidence base is growing for the value of informatics, especially when systems are implemented properly.

References
1. Romano, M. and Stafford, R. (2011). Electronic health records and clinical decision support systems: impact on national ambulatory care quality. Archives of Internal Medicine, 171: 897-903.
2. Mohan, V. and Hersh, W. (2011). EHRs and health care quality: correlation with out-of-date, differently purposed data does not equate with causality. Archives of Internal Medicine, 171: 952-953.
3. Cebul, R., Love, T., et al. (2011). Electronic health records and quality of diabetes care. New England Journal of Medicine, 365: 825-833.
4. Persell, S., Kaiser, D., et al. (2011). Changes in performance after implementation of a multifaceted electronic-health-record-based quality improvement system. Medical Care, 49: 117-125.
5. Classen, D. and Bates, D. (2011). Finding the meaning in meaningful use. New England Journal of Medicine, 365: 855-858.