Honorable Mention

Who should be the next National Coordinator for Health Information Technology (HIT), i.e., the Director of the Office of the National Coordinator for HIT (ONC)? My name appeared recently on a list of 24 individuals nominated as a potential replacement for the current National Coordinator, Dr. David Blumenthal, who is leaving the post next month to return to academia at Harvard University.

Well, the results of the voting are in, and I was flattered to finish fifth, capturing 5.6% of the 736 votes cast. The winner of the poll was Jessica Kahn, Technical Director for HIT for Medicaid at the Centers for Medicare and Medicaid Services. Following in a close second was Dr. Marc Chasin. Vice President and Chief Medical Informatics Officer at St. Luke's Health System in Boise, Idaho. I am delighted to report that Dr. Chasin is a former student of mine, having taken the 10x10 ("ten by ten") course.

I have to admit that I am ordinarily unswayed by magazine or Web polls that are completely unscientific and really just popularity contests. Still, I was flattered to be part of this. Perhaps, as they say in show business, any publicity is good publicity.

This all said, I don't think I will be leaving Oregon Health & Science University any time soon. I have waited my whole career for the situation I and the department I lead are currently in, seeing the maturation of our field and the resources now available to support education and research endeavors within it.

I also do not envy the person who actually replaces Dr. Blumenthal. Clearly the HITECH Act has, to use the terminology of Gartner Hype Cycle, hit its peak of expectations. Some of the programs are down in the trough of disillusionment, although I am confident that most if not all of them will eventually level off in the plateau of productivity. Given the current political state of Washington, DC, where scoring political points seems to have overtaken governing and producing value from government programs, the next National Coordinator is likely to spend a good deal of time in non-productive Congressional hearings. It's not that I don't think government bureaucrats, like everyone else, need to be held accountable, but it is unlikely the primary purpose of those hearings will be to report on the value to healthcare and economy that HITECH has wrought.

Being the optimist that I am, I am not dwelling too much on the current poisoned atmosphere in Washington, DC. I will certainly defend to anyone the productive investment that has been made the federal government in HIT. In our programs funded by educational grants, skills and leadership have been imparted on a new cadre of individuals, and the curricular materials we are producing will have a lasting impact on the primary goal of biomedical and health informatics, which is to improve human health, healthcare, biomedical research, and public health with information.

Natural Language Processing: A Dream That Won't Die … and Shouldn't

One of the longest-standing dreams of informatics, dating back to the early (i.e, 1960s) era of artificial intelligence, is the use of natural language processing (NLP) to extract data about patients from clinical narrative data (e.g., progress notes, discharge summaries, etc.) in the electronic health record (EHR). The notion that you can take the narrative language of clinicians and turn it into concrete facts that can be used for clinical decision support, clinical research, quality measurement, surveillance, etc. is immensely appealing.

Alas, that dream, at least in a generalizable way, is still a dream. You can count a number of my published papers over the years as a few among the many valiant efforts. Unfortunately, the variability (or some might say mangling, especially by physicians) of language, along with the hidden context and meaning "between the lines," makes NLP a very difficult task to program in a computer.

Some, however, have managed to succeed in focused ways. For example, generalizable decision support also never succeeded but it has been found that focused decision support works quite well and is used in EHRs daily. Likewise, there have a number of focused areas where NLP has provided useful data for clinical processes.

It is in this context that I am pleased to report on another contribution to the literature of clinical NLP, which is a paper that appeared in a recent issue of the Journal of the American Medical Informatics Association (JAMIA) and is lead-authored by a former student, Mary Stanfill [1]. I am a co-author. It is always a thrill to see a student publish a peer-reviewed paper, especially one that started as a term paper in one of my courses, advanced to a capstone project in a master's degree, and ultimately ended up in one of the leading journals in our field.

This paper also makes a valuable contribution of being a systematic review of all studies that report results of "automated coding and classification." The analysis shows that there have been many efforts performed using many methods in a variety of clinical domains, with a wide range of results. Of course, this gets to a gripe I have had with clinical NLP and related text mining researchers over the years, which is that evaluation studies have not advanced much beyond measuring the accuracy (e.g., recall, precision, sensitivity, specificity, etc.) of how well systems identify concepts in the text [2]. I would prefer to see the next step in systems being evaluated, such as how well NLP can impact the tasks it might be used for, such as quality measurement programs or facilitating clinical research studies. This would be akin to the "task-oriented" studies of information retrieval systems I performed years ago, which focused on how well searchers completed tasks using retrieval systems rather than just measuring how many relevant articles they retrieved [3].

The good news is that systems using NLP are starting to be deployed in operational clinical settings or clinical and translational research programs, and there is an ever-increasing amount of real data in electronic form for them to use. In addition to a growing number of individual studies, there are also large-scale projects of which NLP is a significant part. There include:

• Informatics for Integrating Biology and the Bedside (i2b2) - a long-standing project to facilitate the use of clinical data for genomic and clinical research. One of its activities includes a yearly challenge evaluation that allows research to compare systems and results on a common task. The i2b2 challenge has looked at automatic de-identification [4], identification of smoking status [5], recognition of obesity and co-morbidities [6], and extraction of medication information [7].
• Electronic Medical Records and Genomics (eMERGE) Network - a multi-center project focused on the use of data in EHRs to facilitate the study of how genetic variability contributes to health and disease [8]. One of the foci includes the use of NLP for extracting data from clinical narratives and integrating it with other data in the clinical record. One accomplishment of this research to date has been the ability to replicate four of seven known gene-disease associations [9].
• SHARP 4 - one of the four collaborative research centers being funded under the HITECH Program to facilitate meaningful use of EHRs, with a focus on secondary use of EHR data.

Another development is the launching this year of a medical records track in the Text Retrieval Conference (TREC) annual information retrieval challenge evaluation. The track will use de-identified records developed by Wendy Chapman and colleagues.

It is also impossible to discuss this topic without acknowledging the discussion around the IBM Watson question-answering system, which recently proved its mettle in a television game show Jeopardy match [10]. IBM has announced some research partnerships that will apply Watson to medical data. This is an interesting research area, but we will need to see real research results to back up the hype [11].

While there are still challenges for clinical NLP, I believe we are seeing a convergence of new methods coupled with growing needs to make use of the increasing volume of clinical data as well as our desire to facilitate re-use of that data for many purposes, such as clinical decision support, quality measurement and improvement, clinical research, and public health reporting and surveillance. While there may be generalizable approaches yet to be discovered, I suspect that evolution will be much like clinical decision support, which has been more successful when engineered to specific domain areas. But as we have also seen with clinical decision support, the ability to perform those specific tasks successfully will be highly valuable to healthcare.

References

1. Stanfill, M., Williams, M., et al. (2010). A systematic literature review of automated clinical coding and classification systems. Journal of the American Medical Informatics Association, 17: 646-651.
2. Hersh, W. (2005). Evaluation of biomedical text mining systems: lessons learned from information retrieval. Briefings in Bioinformatics, 6: 344-356.
3. Hersh, W., Crabtree, M., et al. (2002). Factors associated with success for searching MEDLINE and applying evidence to answer clinical questions. Journal of the American Medical Informatics Association, 9: 283-293.
4. Uzuner, O., Luo, Y., et al. (2007). Evaluating the state-of-the-art in automatic de-identification. Journal of the American Medical Informatics Association, 14: 550-563.
5. Uzuner, O., Goldstein, I., et al. (2008). Identifying patient smoking status from medical discharge records. Journal of the American Medical Informatics Association, 15: 14-24.
6. Uzuner, O. (2009). Recognizing obesity and comorbidities in sparse data. Journal of the American Medical Informatics Association, 16: 561-570.
7. Uzuner, O., Solti, I., et al. (2010). Extracting medication information from clinical text. Journal of the American Medical Informatics Association, 17: 514-518.
8. McCarty, C., Chisholm, R., et al. (2010). The eMERGE Network: a consortium of biorepositories linked to electronic medical records data for conducting genomic studies. BMC Genomics, 4(1): 13. http://www.biomedcentral.com/1755-8794/4/13.
9. Denny, J., Ritchie, M., et al. (2010). PheWAS: Demonstrating the feasibility of a phenome-wide scan to discover gene-disease associations. Bioinformatics, 26: 1205-1210.
10. Ferrucci, D., Brown, E., et al. (2010). Building Watson: an overview of the DeepQA Project. AI Magazine, 31(3): 59-79. http://www.aaai.org/ojs/index.php/aimagazine/article/view/2303.
11. Anonymous (2011). IBM to Collaborate with Nuance to Apply IBM’s "Watson" Analytics Technology to Healthcare. Armonk, NY, IBM. http://www-03.ibm.com/press/us/en/pressrelease/33726.wss.

PCAST Report: What's the Big Deal?

As anyone who works in informatics knows, there is unremitting stream of reports, white papers, blog entries, and other writings from various government agencies, non-profit organizations, consultants, research organizations, and others involved in health information technology (HIT). Some of these reports promote various points of view, including policy directions, while others present interesting ideas to read. (Some do neither!)

One recent report has garnered more attention than any in the last several months (perhaps since the release of the meaningful use rules). This is of course the recent report from the President's Council of Advisors on Science and Technology (PCAST) entitled, Realizing the Full Potential of Health Information Technology to Improve Healthcare for Americans: The Path Forward, which was released in December, 2010. This report, called the “PCAST report” by many, has gained high visibility, perhaps reflecting its origin from the White House. It has led the Office of National Coordinator for HIT (ONC) to ask its HIT Policy Committee to create a workgroup tasked with collecting and analyzing public comments and making recommendations relative to current and future ONC activities.

The report states its case by noting that current HIT systems do not meet their potential, mainly due to the lack of interoperability that results from proprietary data stores (mostly in proprietary systems) that blunt the free flow of data. This is hardly new. The report goes on to advocate what it views as the critical solution to the problem, which is the development of a “universal exchange language” (abbreviated by some though not in the report as “UEL”) based on the notion of data elements being reduced to their atomic core. It advocates that each of these core elements have metadata (“data about data”) tagging that includes the element and its value along with an identity of the patient, a patient-controlled privacy designation, and other provenance information about the element.

The report has certainly generated a great deal of discussion, with most of the major organizations involved in HIT having weighed in during the ONC comment period. The report certainly piqued my interest, since I have always held the view that data is the most critical aspect of everything we do with HIT. I agree with Dr. Blumenthal that data is the “lifeblood” of medicine, and my informatics nirvana consists of data freely flowing between systems and the appropriately authorized people to use it. In my dream world, one could be in one EHR and “Save as…” the data for loading into another EHR or some other application. The data would be so interoperable that any application would recognize documents (e.g., discharge summaries or progress notes), measurements (e.g., vital signs or lab values, and structure data (e.g., prescriptions).

So where does PCAST fit into all of this? In this posting, I will review the PCAST Report, summarize the commentary and criticisms, and give my own further analysis to get others thinking (as a good educator should!).

The report begins with the usual accolades for how HIT has the potential to transform healthcare. In addition to the usual improving clinician access to patient data and decision support, involving patients in their care, and enabling public health and clinical research, the report also notes that HIT can create new markets and jobs for technology as well as support healthcare reforms, including economic changes in the system. It lays out nine use cases that benefit patients, clinicians, public health, and clinical research.

However, the report notes, HIT has fallen short of its promise for four reasons. First, most current systems are proprietary applications not easily adapted into clinical workflow, with proprietary data formats not easily exchangeable. Data is not easily disaggregated or re-aggregated. Second, most healthcare organizations focus on the internal value of EHRs and have no incentive for secondary or external uses of their data for patients, other healthcare organizations, public health, or research. A third reason is that patients have concerns about the privacy and security of information of their data. Finally, the report notes that HIT has been largely focused toward administrative functions and not on improving healthcare.

The UEL will require a common infrastructure for locating and assembling data elements of a patient’s records via secure data element access services (DEAS). Data would remain local, and DEAS would be distributed, intercommunicating, and interoperable. A single appropriately authorized query would be able to locate and assemble a patient’s record across multiple DEAS.

The essential core of the report is Chapters 4-5 (pp. 39-52), which deal with the core of the technology and privacy. Chapter 4 asserts that the approach of applications as “services” does not scale up. (Though the authors seem to violate the notion of separating the application and the data.) It is argued that a better approach for healthcare data is the UEL, which is coded in (eXtensible Mark-up Language) XML and tagged with three metadata elements (in addition to the data and its value):

• Identifying information about patient – including information enabling location of the data (not necessarily a universal identifier)
• Privacy protection information – who may access the data, for what purposes, and either in an identified or de-identified state
• Provenance of data – date, time, equipment used to collect data, personnel who collected it, etc.

The UEL would aim for semantic interoperability. While adherence to specific controlled terminology sets would not be required, it would be strongly encouraged. This would allow over time for data to truly be represented in a universal way.

DEAS activities would include “crawling, indexing, security, identity, authentication, authorization, and privacy.” Queries would be issued against all DEAS on the Internet, and results could be re-constituted into a complete picture of patient. Governments, healthcare organizations, and others would operate the DEAS. In some way, this process would act like Web search engines, although they would need to have very high recall and precision to insure all the appropriate data was retrieved, while incorrect data was not. In conclusion, the chapter claims the UEL is extensible, extractable by middleware, and will lead to innovative uses and applications.

Chapter 5 lays out the privacy and security aspects of the UEL and DEAS. In essence, each and every data element would have a patient-controlled privacy attribute, allowing access to the element and its use in identified or de-identified scenarios. All data would be encrypted, which would not only protect security, but also insert a mechanism to audit access.

A number of leading HIT organizations took the opportunity of the ONC comment period to state their positions. While all applauded the raising of awareness of issues related to data and its interoperability, they also raised a number of criticisms. Many advocated that the PCAST Report serve as a broader vision rather than holding concrete solutions.

The comments about the report can be summarized as follows:

1. The constellation of current standards, as imperfect as they are, meet many of the data-related goals laid out by the report.
2. Many of the ideas of the report, while interesting and worthy of further research, are untested. Having them be the drivers of Stage 2 of meaningful use would be a substantial change in direction and put the larger HITECH Program at risk.
3. Clinical data has context, and reducing its entirety to atomic elements has the potential to lose that context. Re-constituting it may not be possible if that context is lost.
4. Much of the context in clinical data requires that records be more document-centric or at least structured in groups of elements. Disaggregating documents could lose the context they provide.
5. While everyone agrees that structured data is most desirable, some data in healthcare is too nuanced, and unstructured text is required to describe it.
6. While industry-wide standards are important, no industry with data as complex as healthcare has tried an approach like this.
7. The notion of setting privacy at the individual element is highly problematic. Allowing the patient to choose which elements can be seen or not seen by clinicians, researchers, or others could introduce many unintended consequences. It would be preferable for the patient to specify general privacy policies that are then referenced by the data elements.
8. Patients’ views of privacy may change over time, as diseases change and their own disease course changes.
9. Search engines are imperfect. The DEAS would need to operate at high levels of recall and precision that are unprecedented for Internet-based search mechanisms.
10. While the report correctly notes that current HIE efforts are struggling, it ignores that major reasons for this, which have more to do with the lack of a business model for HIE than anything about the technology.

A number of specific comments from these organizations are poignant:

• American Hospital Association (AHA) - The report should set a broader vision rather than focus on concrete solutions. Setting privacy at element level could fracture the patient record. Tagging each piece of data could be costly and inefficient. DEAS are likely to face same challenges as HIEs, with lack of a business model. ONC should change policy direction for Stage 2 of meaningful use only with great caution.
• Radiological Society of North America (RSNA)/American College of Radiology (ACR) - Echoed many of the same comments and noted we need a uniform method to manage patient identity.
• Integrating the Healthcare Enterprise (IHE) USA - Noted that the current IHE profiles cover most of the functionality required for the 9 use cases.
• Healthcare Information and Management Systems Society (HIMSS) -Privacy is contextual and changing, especially as diseases change and information about them becomes less sensitive. Encryption of the data elements provides security and an audit trail but can adversely impact workflow. The objectives of the report might not be possible without a universal patient identifier. By atomizing data, we run risk of data becoming dissociated and not being able to detect errors, so any grouping in the source data should be maintained. Metadata tagging should be virtual and not physical. Tags should be referenced and not attached, since some (e.g., privacy) might change over time. Data elements separated from documents and records potentially robs them of their context.
• HIMSS Electronic Health Record Association (EHRA) -It is better to tag data on document or record level. The privacy approach is potentially unworkable. A large-scale effort of this approach is untested.
• AMIA - Chapter 4 provides general ideas but no details nor references. There is no evidence that this approach will lead to improved care. The report was for the most part silent about other federal agencies, especially the National Library of Medicine, which has great expertise in some aspects of the proposed approach, especially related to terminology development and usage. The report underestimates the complexity of modeling the domain of medicine. It ignores past failed efforts along similar lines, such as the caBIG caDSR project. The DEAS may not be scalable or practical. ONC should not deviate from already tight timeline of Stage 2 of meaningful use. We can learn lessons from the slow adoption of HL7 Version 3, which is not suited to efficient description of task information models. There is too much focus on healthcare and not enough on health.

Other organizations that weighed included:

• HL7
• Federation of American Hospitals (FAH)
• Center for Democracy in Technology
• Project HealthDesign
• Markle Foundation
• Clinical Groupware Collaborative
• Faster Cures
• Society for Participatory Medicine
  

And of course, a number of bloggers had things to say. As always, John Halamka provided great early summaries of the report and the deliberations of the ONC HIT Policy Committee:
http://geekdoctor.blogspot.com/2010/12/spirit-of-pcast.html
http://geekdoctor.blogspot.com/2011/01/primer-on-xml-rdf-json-and-metadata.html
http://geekdoctor.blogspot.com/2011/01/example-for-pcast.html
http://geekdoctor.blogspot.com/2011/01/general-principles-of-universal.html

In a widely cited posting, Wes Rishel noted some critical points: Information flow for patient care occurs at the level of documents. Taking elements out of larger context can lose context. PCAST data elements are actually molecules, not atoms. There are plenty of molecule definitions, these should be used. Another well-known blogger, Keith Boone, added that a good deal of what the report hopes to accomplish can be done with existing standards.

What will be the impact of the PCAST report? We will find out for sure in April when the ONC releases its analysis and plans for incorporating the report’s ideas and proposals. If nothing else, the report has led to increased discussion about the importance of data interoperability, which even its critics applaud. My hope is that there is at least an acceleration toward the vision of interoperable data that most in informatics share.

Some Supplemental Information from the PCAST Report

The nine use cases were lumped into three categories based upon to whom they provided value.

Value to patients:

• Patient on warfarin wanting to know if it is safe to take an NSAID drug for an injury.
• Woman with lung mass newly discovered in a community hospital referred to a large academic medical center.

Value to clinicians:

• Internist developing primary care medical home.
• Small practice with clinicians sharing records and communicating with patients via email.
• Cardiology clinics in a part of country collaborating to improve care for patients with recent myocardial infarction.
• Family physician embedding alerts in practice to improve preventive care.

Value to research and public health:

• Physician caring for a patient enrolled in a national clinical trial.
• FDA carrying out post-marketing surveillance of adverse reactions to drugs.
• Communities or states measuring improvement toward health goals.

The final published conclusions of the report were:

1. HHS and ONC have laid a foundation for progress under meaningful use and HITECH.
2. Achievement of goals for healthcare involves accelerated progress toward robust health information exchange.
3. Effort should now focus on development of a universal exchange language that enables data to be shared and re-assembled across institutions, subject to strong privacy safeguards based on patient privacy preferences.
4. Creating these requirements is technically feasible.
5. ONC should move rapidly to develop these capabilities for stages 2 and 3 of meaningful use.
6. CMS should modernize and restructure its IT platforms to serve as a driver for progress in health IT.