Reaffirming the Value of Health Information Technology

The year 2013 was not a great year for health information technology (HIT), punctated by events happening in the beginning, middle, and end of the year. The year started with a commentary in the journal Health Affairs, noting that the promises that drove the purported benefits for HIT in the US federal HIT investment were still unmet [1]. This came on the heels of the IOM report in late 2012 raising concerns about HIT safety [2]. In the middle of the year, growing concern rose over the ability of healthcare organizations to meet the various government regulatory requirements (typified by a news release from the American College of Physicians), from the meaningful use criteria to ICD-10. Late in the year, while not HIT per se, was the Healthcare.gov debacle, which tarnished the general image of IT for healthcare-related activities, especially those led by government agencies.

This new year has started better, in particular with the publication of an update of a systematic review that reaffirms the value, with some caveats, of what we are trying to accomplish in HIT. Systematic reviews do not report new primary research data, but instead synthesize all of the existing research to catalog and summarize our knowledge (sometimes generating new knowledge in the process through techniques like meta-analysis) as well as identify gaps that may exist. This week saw the publication of a systematic review that was the latest update in a series of systematic reviews about HIT dating back to 2006. The bottom line result was that the proportion of studies reporting beneficial results for HIT continues to be high, varying from 51-73% in the different categories of use that were assessed. When studies reporting mixed or neutral results were factored in, the proportion finding negative results only were small, varying from 0-16%. The functions most commonly assessed in the studies were clinical decision support and computerized provider order entry. A figure from a posting in the ONC Health IT Buzz Blog depicts this all graphically quite well.

There is another important finding reported in this analysis, which is that about half of all studies were done using commercial HIT systems. One of the most important "findings" of the earlier iterations of this systematic review was a preponderance of studies from a small number of "HIT leader" institutions, mostly using home-grown electronic health record systems. This always raised some concerns about generalizability, i.e., could other institutions without the resources of such institutions replicate what the leaders were able to accomplish.

The authors also noted some caveats to the evidence base and provided a set of recommendations for improving it going forward. One caveat is the limited nature by which systems have been studied, usually done as before-after analyses and often not providing adequate contextual description for the results. The authors also note a potential for publication bias, both in studies that may not be reported at all to adverse consequences that are not adequately measured or described in those that are published. I heartily concur with the authors' recommendations that future studies include more robust experimental designs and a focus on those that provide better understanding of the context of the findings and how they provide value for health and healthcare delivery.

Nonetheless, this new systematic review reaffirms the scientific evidence for work in informatics going forward. We still must try to build systems that meet the broadest needs and provide value, but also evaluate them with enlightened skepticism. I am also encouraged by the growing number of studies using commercial HIT systems and not taking place in informatics exemplar institutions. This affirms the value of the approach we are taking in the academic program at OHSU, which is to build a well-trained informatics workforce in our educational program to disseminate and apply the best science and practice. It also demonstrates the value of new partnerships we are establishing, such the one launched between OHSU and Epic that will allow research and education on top of a state-of-the-art commercial platform.

References

1. Kellermann, AL and Jones, SS (2013). What will it take to achieve the as-yet-unfulfilled promises of health information technology? Health Affairs. 32: 63-68.
2. Anonymous (2012). Health IT and Patient Safety: Building Safer Systems for Better Care. Washington, DC, National Academies Press.
3. Jones, SS, Rudin, RS, et al. (2014). Health information technology: an updated systematic review with a focus on meaningful use. Annals of Internal Medicine. 160: 48-54.