Annual Reflections at the End of 2015

As regular readers of this blog know, I traditionally end each year with a posting reflecting back on the past year. While this year has been another great success for myself and our informatics program at Oregon Health & Science University (OHSU), it has been somewhat of a transitional year for the informatics field. Many of the new and exciting initiatives in the informatics field from recent years are no longer novel, with some now settling into “midlife” and others being called out for retirement.

One program settling into midlife, although being called out for retirement by many, has been the Health Information Technology for Economic and Clinical Health (HITECH) Act. The launching of this blog, and indeed the catapult to much larger visibility of the informatics field, owes a great deal to HITECH. There is no question that HITECH has succeeded on some levels, at least in terms of increasing electronic health record (EHR) adoption, as I have noted before. A recent report from the Commonwealth Fund confirms what statistics from the Office of the National Coordinator for Health IT (ONC) show: the US is no longer a world laggard in health IT and is in some ways a global leader [1].

But there is no question that not all with HITECH has gone well. Despite the widespread adoption of EHRs, they are still very imperfect [2]. At best, they impede clinician workflow and at worst, they cause some of the safety problems they have been touted to rectify. And one vision has clearly not been achieved, which is interoperable data across systems, even those from the same vendor [3]. Going forward, the informatics field must provide leadership to guide the best use of EHRs and related systems, which is spelled out excellently in the AMIA EHR-2020 Task Force white paper [4].

Another interesting happening, perhaps related to health IT achieving midlife, is that the quantity of health IT blogging seems to be tapering off. In this blog for example, I had fewer posts this year than any since the first year I started the blog. The same is true for a number of other well-known health IT bloggers, such as Keith Boone and John Halamka. I do not view this as necessarily a bad thing, but perhaps just an indicator that some of the formerly novel aspects of informatics are reaching maturity, and there is less to say on a day-to-day basis.

Also a continuing happening this year was the continued growth of data science, and confusion as to its relationship to informatics. Informaticians are not the only ones expressing confusion where they belong in this new field; statisticians are feeling the same [5]. Nonetheless, there is no question that data and learning from it will drive many scientific fields going forward.

I would also like to call out some other year-end posts from some other bloggers, namely John Halamka, for recapping 2015 overall plus adding some focus on security and looking ahead to 2016, and the folks at HISTalk, who have a comprehensive list of 2015 top stories and 2016 predictions.

On a personal and program level, this year had a number of achievements. I was honored to be bestowed the HIMSS Physician IT Leadership Award. I was also awarded a new grant to update the ONC Health IT Curriculum. On a program level, the OHSU Department of Medical Informatics & Clinical Epidemiology (DMICE) launched its new clinical informatics fellowship and continued its mutli-faceted success in its major missions of research and education.

Looking ahead to 2016, there are plenty of new projects and other activities to keep myself and our department busy. It will be interesting to see how HITECH fares and how the critical need for data interoperability evolves. And of course, new opportunities will emerge for myself and DMICE, many of which cannot even be foreseen now.

References

1. Osborn, R, Moulds, D, et al. (2015). Primary care physicians in ten countries report challenges caring for patients with complex health needs. Health Affairs. 34: 2104-2112.

2. Rosenbaum, L (2015). Transitional chaos or enduring harm? The EHR and the disruption of medicine. New England Journal of Medicine. 373: 1585-1588.

3. Anonymous (2015). Connecting Health and Care for the Nation: A Shared Nationwide Interoperability Roadmap version 1.0 (Roadmap). Washington, DC, Department of Health and Human Services. https://www.healthit.gov/sites/default/files/hie-interoperability/nationwide-interoperability-roadmap-final-version-1.0.pdf.

4. Payne, TH, Corley, S, et al. (2015). Report of the AMIA EHR-2020 Task Force on the status and future direction of EHRs. Journal of the American Medical Informatics Association. 22: 1102-1110.

5. Donoho, D (2015). 50 years of Data Science. Princeton NJ, Tukey Centennial Workshop. https://dl.dropboxusercontent.com/u/23421017/50YearsDataScience.pdf.

Volume is Only One of the Four "V"s of Big Data, Especially for the Right Data

One widely accepted definition of Big Data is that it entails four “V”s: volume, velocity, variety, and veracity. In other words, Big Data is defined by there being a great deal of it (volume), coming at us rapidly and continuously (velocity), taking many different forms and types (variety), and originating from trustworthy sources (veracity). Among some people, however, there seems to be more focus on one of the Vs above all others, namely volume. I suppose that is not surprising, given that the adjective qualifying the noun head in Big Data is one that describes size.

However, as I and others have written over the years, there are many aspects of data that are just as important as its quantity. Even worse, I have heard many people imply in their statements about data science that you cannot do real data science without massive amounts of data, in turn requiring massive amounts of storage capacity and computer power (also costing much money).

Make no mistake, we do need to consider the volume aspects of data when discussing data science. But we must not lose in the discussion what we hope to accomplish with the data, which one writer refers to as the fifth V of Big Data, namely value [1]. Sometimes value emanates from harnessing the size of a data set, but other times the veracity or variety take on more importance.

I have written about the importance of value as well, noting that meaningless correlations with large amounts of data do not really mean much of anything, and that data scientists must also understand basic research principles, such as causality. So yes, let us prepare for a future where we leverage Big Data to improve health, biomedicine, and other important societal needs, but we also need to remember that we do not always need massive amounts of data, especially that whose veracity we may not know, to derive other value. Perhaps akin to the “rights” of clinical decision support [2], the best data science is more about having access to the right data using the right amount of data at the right time.

References

1. Marr, B (2015). Why only one of the 5 Vs of big data really matters. IBM Big Data & Analytics Hub. http://www.ibmbigdatahub.com/blog/why-only-one-5-vs-big-data-really-matters.
2. Osheroff, JA, Teich, JM, et al. (2012). Improving Outcomes with Clinical Decision Support: An Implementer's Guide, Second Edition. Chicago, IL, Healthcare Information Management Systems Society.

New NIH Biosketch Allows Better Documentation of Contributions to Science

One of the most important documents for a US-based biomedical researcher is the National Institutes of Health (NIH) Biosketch. This short document summarizes the accomplishments of a scientist apply for an NIH grant, listing his or her job positions, educational history, a summary of key publications, and a listing of current grant funding. The NIH Biosketch is also often used as a summary of one’s larger curriculum vitae (CV).

NIH has tweaked the Biosketch over the years, and the most recent update provides an excellent approach that allows researchers to not just summarize their most prominent publications, but also to give a statement about them in the context of the individual's contributions to science. For each contribution, he or she can provide up to four key publications for each. I enjoyed the exercise of updating my Biosketch to the new form, and thought it would be worthwhile to reproduce the scientific contributions and key publications here.

1. My initial research focused on the development and implementation of information retrieval (IR, also called search) systems in biomedicine and health. I experimented with concept-based approaches to indexing and retrieval of knowledge-based information. Subsequently, I found that methods for evaluation systems were inadequate, and developed an interest in new approaches to evaluation. My interests in search have also evolved with the emergence of new content for retrieval, such as medical images and electronic health record data, especially textual notes in the latter.

• Hersh WR, Greenes RA, SAPHIRE: an information retrieval system featuring concept matching, automatic indexing, probabilistic retrieval, and hierarchical relationships, Computers and Biomedical Research, 1990, 23: 410-425.
• Hersh WR, Crabtree MK, Hickam DH, Sacherek L, Friedman CP, Tidmarsh P, Moesbaek C, Kraemer D, Factors associated with success for searching MEDLINE and applying evidence to answer clinical questions, Journal of the American Medical Informatics Association, 2002, 9: 283-293. PMC344588.
• Hersh W, Kalpathy-Cramer J, Müller H, The ImageCLEFmed medical image retrieval task test collection, Journal of Digital Imaging, 2009, 22: 648-655.
• Hersh W, Voorhees E, TREC Genomics special issue overview, Information Retrieval, 2009, 12: 1-15.

2. My interest work in IR has converged with another interest in the secondary use of clinical (especially electronic health record) data. I have made contributions not only in attempting to leverage such data, but also addressing caveats and recommendations for its use.

• Voorhees E, Hersh W, Overview of the TREC 2012 Medical Records Track, The 21st Text Retrieval Conference - TREC 2012.
• Edinger T, Cohen AM, Bedrick S, Ambert K, Hersh W, Barriers to retrieving patient information from electronic health record data: failure analysis from the TREC Medical Records Track, Proceedings of the AMIA 2012 Annual Symposium, 2012, 180-188, PMC3540501.
• Hersh WR, Weiner MG, Embi PJ, Logan JR, Payne PR, Bernstam EV, Lehmann HP, Hripcsak G, Hartzog TH, Cimino JJ, Saltz JH, Caveats for the use of operational electronic health record data in comparative effectiveness research, Medical Care, 2013, 51(Suppl 3): S30-S37. PMC3748381.
• Hersh WR, Cimino JJ, Payne PR, Embi PJ, Logan JR, Weiner MG, Bernstam EV, Lehmann HP, Hripcsak G, Hartzog TH, Saltz JH, Recommendations for the use of operational electronic health record data in comparative effectiveness research, eGEMs (Generating Evidence & Methods to improve patient outcomes), 2013, 1:14.

3. I have also made contributions in conducting systematic reviews of evaluative research of informatics technologies. These reviews can be challenging because many evaluations use weak evaluation methodologies, in part because these technologies are tools rather than typical medical tests or treatments.

• Hersh WR, Hickam DH, How well do physicians use electronic information retrieval systems? A framework for investigation and systematic review, Journal of the American Medical Association, 1998, 280: 1347-1352.
• Hersh WR, Hickam DH, Severance SM, Dana TL, Krages KP, Helfand M, Diagnosis, access, and outcomes: update of a systematic review on telemedicine services, Journal of Telemedicine and Telecare, 2006, 12(Supp 2): 3-31.
• Stanfill MH, Williams M, Fenton SH, Jenders R, Hersh W, A systematic review of automated clinical coding and classification systems, Journal of the American Medical Informatics Association, 2010, 17: 646-651. PMC3000748.
• Hersh W, Totten A, Eden K, Devine B, Gorman P, Kassakian S, Woods SS, Daeges M, Pappas M, McDonagh MS, Outcomes from health information exchange: systematic review and future research needs, JMIR Medical Informatics, 2015, 3(4): e39.

4. Being the leader of a major biomedical informatics educational program, I have also carried out research characterizing the informatics professional workforce. My study on the need for health IT professionals played a role in workforce development being a component of the Health Information Technology for Clinical and Economic Health (HITECH) Act of the American Recovery and Reinvestment Act (ARRA).

• Hersh W, Who are the informaticians? What we know and should know, Journal of the American Medical Informatics Association, 2006, 13: 166-170. PMC1447543.
• Hersh W, Wright A, What workforce is needed to implement the health information technology agenda? Analysis from the HIMSS Analytics™ Database, Proceedings of the AMIA 2008 Annual Symposium, 2008, 303-307. PMC2656033.
• Hersh W, The health information technology workforce: estimations of demands and a framework for requirements, Applied Clinical Informatics, 2010, 1: 197-212. PMC3632279.
• Hersh WR, Margolis A, Quirós F, Otero P, Building a health informatics workforce in developing countries, Health Affairs, 2010, 29: 274-277.

5. Also as a result of being an educational leader, I have carried out evaluation of educational programs in informatics, including those using distance learning technologies.

• Hersh W, Williamson J, Educating 10,000 informaticians by 2010: the AMIA 10x10 program, International Journal of Medical Informatics, 2007, 76: 377-382.
• Hersh WR, A stimulus to define informatics and health information technology, BMC Medical Informatics and Decision Making, 2009, 9: 24.
• Otero P, Hersh W, Luna D, Quirós F, Translation, implementation and evaluation of a medical informatics distance-learning course for Latin America, Methods of Information in Medicine, 2010, 49: 310-315.
• Mohan V, Abbott P, Acteson S, Berner ES, Devlin C, Hammond WE, Kukafka R, Hersh W, Design and evaluation of the ONC health information technology curriculum, Journal of the American Medical Informatics Association, 2014, 21: 509-516.

NIH now also allows scientists to create a complete list of published work in the MyBibliography section of the NCBI Web site.

The Evidence Base for Health Information Exchange

One of my major projects over the last couple years has been a systematic review of the research that has been conducted on health information exchange (HIE). I wrote about this project when it first started and when our protocol for the review was posted for public comment. The report was funded by the Evidence-Based Practice Centers Program of the Agency for Healthcare Research and Quality (AHRQ). While the review itself has been done for several months, we have been finalizing the report and publications derived from it since then. I am pleased to report that both the complete report [1] plus a paper reporting on the outcomes from studies of HIE [2] have now been published. There will be some additional papers on other aspects of the report as well as a book chapter summarizing the report to be published next year [3].

This report has certainly given me the opportunity to reflect over the last couple years of the state of HIE and the interoperability required to support it. The major finding of the report echoes findings of a similar couple of systematic reviews I led on the topic of telemedicine published in 2001 [4] and 2006 [5], which is that the breadth and quality of the research are limited. There is no question that performing research on HIE is difficult. After all, HIE is not a test or a treatment, but rather a tool that facilitates other aspects of healthcare. Nonetheless, the research base for HIE is limited, and should be improved if we want to discern it benefits and optimal use. The paper provides our recommendations for improving research on HIE outcomes going forward [2].

Our report also gives us an opportunity to think about some of the larger issues around the current role and future directions of HIE. If I had to lament about HIE, I would say that it is an unfortunate requirement at this time for us to need so many different organizations (135 according to the last eHI annual survey of them [6]) devoted to HIE. In the ideal world, there would be no need for HIE organizations, but instead, there would be sufficient interoperability of systems, along with rules and regulations, to allow information to flow seamlessly between appropriate parts of the healthcare system. For example, a physician in his or her office could seamlessly transmit a consultation, receive laboratory results or a discharge summary, or notify a public health department of a reportable event without requiring an HIE entity to facilitate those activities. The information transmitted would be formatted into some standardized form and sent securely to an authenticated site, all facilitated by standard protocols used by the entire industry.

Hopefully the new emphasis of ONC on interoperability [7] and the underlying standards required [8] will facilitate more seamless HIE. While many have argued that the criteria for meaningful use should have placed more emphasis on secure and standardized information exchange rather than specific EHR functionality, such as clinical decision support or specific quality measures, that is all now proverbial water under the bridge. I am certain everyone agrees that we need to focus on seamlessly interoperable health IT going forward. I also hope in the process that robust research is carried out, not only to assess the value of HIE but also determine the best ways to implement it.

An interesting side note to this report is an episode related to another systematic review on HIE that was published in late 2014 [9]. One of our competitors for the contract that was awarded by AHRQ to our institution went out and found another source for funding to carry out a review. Not only did they perform a review that was reduced in scope from our review (it omitted public health and any type of HIE outside the US), but they were also able to bypass all of the processes that AHRQ has to insure the systematic reviews it funds have stakeholder engagement, public comment, and broad peer review. As such, the other group was able to complete their review well in advance of ours and get it published in a very high profile journal, Annals of Internal Medicine. That journal publishes a good number of AHRQ-funded systematic reviews, but understandably did not want to publish ours after they had already published another systematic review on the topic of HIE. While I have no problems with science being competitive in terms of accolades going to the first to publish, I do find it disappointing that another group basically duplicated our review and short-circuited the usual processes of AHRQ.

References

1. Hersh W, Totten A, Eden K, Devine B, Gorman P, Kassakian S, Woods SS, Daeges M, Pappas M, McDonagh MS. Health Information Exchange. Evidence Report/Technology Assessment No. 220. AHRQ Publication No. 15(16)-E002-EF. Rockville, MD: Agency for Healthcare Research and Quality; 2015. http://www.effectivehealthcare.ahrq.gov/ehc/products/572/2154/health-information-exchange-report-151201.pdf.
2. Hersh, WR, Totten, AM, et al. (2015). Outcomes from health information exchange: systematic review and future research needs. JMIR Medical Informatics, 3(4):e39. http://medinform.jmir.org/2015/4/e39/.
3. Hersh, WR, Totten, AM, et al. (2016). The Evidence Base for Health Information Exchange. In Health Information Exchange: Navigating and Managing a Network of Health Information Systems. B. Dixon. Amsterdam, Netherlands, Elsevier, in press.
4. Hersh, WR, Helfand, M, et al. (2001). Clinical outcomes resulting from telemedicine interventions: a systematic review. BMC Medical Informatics and Decision Making. 1: 5. http://www.biomedcentral.com/1472-6947/1/5.
5. Hersh, WR, Hickam, DH, et al. (2006). Diagnosis, access, and outcomes: update of a systematic review on telemedicine services. Journal of Telemedicine & Telecare. 12(Supp 2): 3-31.
6. Anonymous (2014). 2014 eHI Data Exchange Survey Key Findings. Washington, DC, eHealth Initiative, http://visit.medicity.com/rs/aetnainc/images/2014%20eHI%20Data%20Exchange%20Survey%20Results.pdf.
7. Anonymous (2015). Connecting Health and Care for the Nation: A Shared Nationwide Interoperability Roadmap version 1.0 (Roadmap). Washington, DC, Department of Health and Human Services. https://www.healthit.gov/sites/default/files/hie-interoperability/nationwide-interoperability-roadmap-final-version-1.0.pdf.
8. Anonymous (2015). Draft 2016 Interoperability Standards Advisory. Washington, DC, Department of Health and Human Services. https://www.healthit.gov/standards-advisory/2016.
9. Rudin, RS, Motala, A, et al. (2014). Usage and effect of health information exchange: a systematic review. Annals of Internal Medicine. 161: 803-811.