About a year ago there was a big dustup in the informatics field concerning a study published by Romano and Stafford in Archives of Internal Medicine that purported to show that electronic health record (EHR) use was not associated with improved quality of care [1]. As honest informaticians, we need to take such research seriously, aiming to improve what we do based on the evidence. This study, however, was problematic, in that it was based on an older data set not designed for answering questions such as the one asked by Romano and Stafford. A better approach would have been to perform a prospective clinical trial that directly assessed an informatics intervention, one of which was indeed published a few months later that did show improvement in care augmented by use of an EHR [2].
Now comes a similar situation a year later with the publication of a study by McCormick et al. in Health Affairs, which uses the same data source to show that physicians who have access to computerized imaging results (not necessarily via an EHR) have a 40-70% higher likelihood of ordering imaging tests [3]. This study set off a similar conversation about whether we are jumping the gun, especially with regards to the substantial federal investment in EHR adoption through the Health Information for Clinical and Economic Health (HITECH) Act. As with the Romano and Stafford study, this new study set off a lot of debate, including an exchange between the National Coordinator for Health IT and a rebuttal by the authors.
It is unfortunate to have to reiterate that we should be guided by the evidence, but given that many of us do have careers staked on the success of the HITECH Act, we must acknowledge potential biases and be as objective as possible in evaluating research results. That said, the study by McCormick truly uses a very weak methodology and certainly does not justify the sweeping conclusions by the authors in their paper or their rebuttal.
Similar to the Romano and Stafford study, this study makes associations with data sources that are not really designed to answer the question of whether EHRs will reduce test ordering. Of course, an even more fundamental question is whether reduced test ordering is something we desire anyways. While imaging tests are clearly over-utilized in healthcare [4,5], this study is incapable of telling us the value of the imaging that was ordered in increased amounts by physicians with access to electronic results. As such, we have no clue as to whether the imaging may or may not be warranted, or how the increased ordering impacted care of the patients for whom it was ordered. It is entirely possible that the increased imaging was beneficial in the management of those patients.
The authors conclude at the end of their abstract that "use of these health information technologies, whatever their other benefits, remains unproven as an effective cost-control strategy with respect to reducing the ordering of unnecessary tests." This is indeed a broad, sweeping conclusion that is hardly warranted from the methods or results of their study.
The rebuttal by Dr. Mostashari is reasonable, noting that the results of this study tell us little of the value of EHRs and the HITECH investment, which is what the authors seem to want to criticize in their results. This certainly comes out in their rebuttal, where they use a number of adjectives to impugn his motives. I do agree with their call for dialogue as well as well-designed clinical trials to assess the benefits of health IT. It may well be that the results of their research are true, and that EHRs will increase costs by making the ordering of expensive tests easier. But we really need our research to answer the larger questions of the value to patient outcomes. We also need to explore how larger changes in our healthcare system, particularly a reimbursement approach that favors quantity over quality, will be empowered by better information systems. With research focusing on those questions, we will be able to ascertain the true value of health IT and how we can improve our use of it.
References
[1] Romano, M. and Stafford, R. (2011). Electronic health records and clinical decision support systems: impact on national ambulatory care quality. Archives of Internal Medicine, 171: 897-903.
[2] Cebul, R., Love, T., et al. (2011). Electronic health records and quality of diabetes care. New England Journal of Medicine, 365: 825-833.
[3] McCormick, D., Bor, D., et al. (2012). Giving office-based physicians electronic access to patients' prior imaging and lab results did not deter ordering of tests. Health Affairs, 31: 488-496.
[4] Baker, L., Atlas, S., et al. (2009). Expanded use of imaging technology and the challenge of measuring value. Health Affairs, 27: 1467-1478.
[5] Hillman, B. and Goldsmith, J. (2010). The uncritical use of high-tech medical imaging. New England Journal of Medicine, 363: 4-6.
