Information Retrieval (Search) in Health and Biomedicine Still "Springs" Eternal

One of my earliest visions of computers in medicine was the ability to type in a question and get an answer. In 1980s, while everyone in informatics was trying to build expert systems, I followed a different dream, of being able to find clinical information seamlessly. In that decade, however, I never could have imagined being able to pull up something called a Web browser, typing in words, and getting back "pages." Especially as I can do now, with something that fits in my pocket, also makes phone calls, and is connected to something I had not yet heard of in the 1980s (before I started my informatics training) called the Internet.

This fascination guided my early research interests in the area of information retrieval. I write about it now because every spring I teach my course on this topic in the OHSU graduate program, BMI 514/614. (Hence the title of this posting.) My interest in this area resulted in dozens of scientific papers and a textbook, currently in its third edition [1]. Despite the marvel I have for today's modern systems, I always have to ask myself, Why didn't I think of the idea of ranking the output (Web pages) by how many other pages pointed to them? Had I thought of that before a couple Stanford graduate students named Brin and Page, my life might be considerably different. Or at least my wealth!

I suppose one is getting up in the years when you marvel at how things are now relative to how you remember them. I certainly recall "searching" when I was in medical school in the 1980s, which involved thumbing through the giant Index Medicus books on long shelves in the library. You would "link" to the full text by walking to a different part of the library where the journals were. If your needs were really critical, you could call on a librarian for help, who would take your request to a special computer that accessed a database somewhere (which happened to be MEDLINE, from the National Library of Medicine).

I actually did my first on-line searching in the 1980s. I was able to access PaperChase, and later Elhill, through dial-up networks, though at a price. For an even heftier price, you could get access to the full text … at least "text" in monospaced font and no figures or images. The world did advance, and by 1998 you could search Pubmed for free. (Al Gore, who actually deserves more credit in this area than his critics deny him, did the first "free" search.)

Now, of course, searching is ubiquitous. You can't even not do it, since most browsers will throw you into a search engine when you type in an invalid Web address (URL) into your browser. And the world not only searches, but searches for health information. The two major periodic surveys of health information searching show that 80% of Internet users have searched for health information for themselves, their family, or their friends [2, 3].

Of course, like many areas of informatics, while use of systems is ubiquitous, not all of the problems of systems are solved. Indeed, a few years ago I wrote a short piece on this topic [4]. As wonderful as today's search systems are, we still have many areas for improvement. In that paper, I identified four areas where grand challenges remained:

• Content - getting diverse users to the right information for the right task
• Indexing - developing better metadata to get searchers to that proper content
• Linkage - allowing navigation across multiple resources, even those of different publishing entities
• Access - making access as open as possible but still being protective of intellectual property

Just as I could not fathom the World Wide Web in the 1980s, I wonder as I write this in 2011 what the world of search and on-line knowledge access will be a decade or two from now.

References

Hersh, W. (2009). Information Retrieval: A Health and Biomedical Perspective (3rd Edition). New York, NY. Springer.
Fox, S. (2011). Health topics. Washington, DC, Pew Internet & American Life Project. http://www.pewinternet.org/~/media//Files/Reports/2011/PIP_HealthTopics.pdf.
Taylor, H. (2010). "Cyberchondriacs" on the Rise? Those who go online for healthcare information continues to increase. Rochester, NY, Harris Interactive. http://www.harrisinteractive.com/vault/HI-Harris-Poll-Cyberchondriacs-2010-08-04.pdf.
4. Hersh, W. (2008). Ubiquitous but unfinished: grand challenges for information retrieval. Health Information and Libraries Journal, 25(Suppl 1): 90-93.

What is the Evidence Base for Informatics, Health IT, and Related Areas? Some Recent Analyses

The first part of 2011 has brought a number of publications, and subsequent discussion, about the "evidence base" for the efficacy of biomedical and health informatics interventions, including electronic health records. These publications and conversations come against a backdrop of a very poisoned political environment in the United States, where everything about healthcare, including informatics, has become unfortunately very politicized. In this posting, however, I will stick to the science.

The first high-profile study of the year was the on-line posting of the Archives of Internal Medicine paper by Romano and Stafford [1], which I discussed in an earlier posting. The official publication of the paper, as well as letters about it, will be published in May, 2011.

Probably the next most high-profile study was the publication of an update of a systematic review of studies of outcomes from health information technology interventions by Buntin and colleagues [2]. This was actually the second update of an original systematic review that was published in 2006 by Chaudhry and associates [3], the first update of which was published by Goldzweig and colleagues in 2009 [4].

Systematic reviews are comprehensive reviews of all research evidence on a given area or question [5]. When studies are homogeneous enough (e.g., all studies assessing the treatment of hypertension to reduce cardiovascular disease), a mathematical technique known as meta-analysis can be performed to combine results across studies to achieve larger a sample size and more statistical power. But most areas, certainly so in informatics, have research questions too heterogeneous to enable use of meta-analysis. Nonetheless, studies can be categorized to look at general questions asked, such as efficacy of decision support to reduce medical error or access to data in a more timely manner to reduce cost of care.

The three successive systematic reviews [2-4] using relatively similar methodology have summarized outcomes of studies of health information technology (HIT) over particular time periods:

• Chaudhry, 2006 – studies from 1995-2004 [3]
• Goldzweig, 2009 – studies from 2004-2007 [4]
• Buntin, 2011 – studies from 2007-2010 [2]

As with most systematic reviews, these captured a broad net of literature and reviewed it for quality of methodology and its results.

Chaudhry et al. identified 257 studies, with the most benefit shown for:

• Adherence to guideline-based care
• Enhanced surveillance and monitoring
• Decreased medical errors

An interesting caveat of the results that the authors noted was that 25% of the identified studies came from four institutions (Partners Healthcare, Veteran's Administration, Indiana University/Regenstrief Institute, and Vanderbilt University) and there were few studies of commercial systems, raising concerns about generalizability.

In their update, Goldzweig et al. found 179 new studies. They noted comparable results to the study of Chaudhry et al., but also found an increased number of studies of patient-focused applications that ran external to EHR, e.g., Web-based care management. They note a small increase in the number of studies of commercial, off-the-shelf systems, though 20% of studies still came from the four leading institutions. They also found there was still a paucity of cost-benefit analyses.

In the new systematic review, Buntin et al. identified 154 new studies with 278 individual outcome measures. While acknowledging wide divergence of study quality and methodologies, not to mention outcomes studied, they noted that 96 (62%) of studies had positive improvement in one or more aspects of care, with 142 (92%) showing positive or mixed positive-negative outcomes. They found that the studies used quantitative and qualitative approaches, with those using statistical hypothesis testing more likely to have positive outcomes. They slightly redefined “health IT leader” institutions, but noted that a large number (28) still came from these institutions, but did decreased somewhat to 18% of the studies. Somewhat reassuring  was that the “leader” studies did not differ in methods or results from the other studies.

Buntin et al. grouped the outcomes into seven categories, noting document improvement in all of them:

• Access to care
• Preventive care
• Care process
• Patient satisfaction
• Provider satisfaction
• Effectiveness of care
• Efficiency of care

Another bit of evidence from early 2011 was a review of all eHealth systematic reviews took exception to direction and quality of evidence [6]. The authors note that many studies of eHealth, including clinical applications (i.e., health IT), had poor methodology, raising concern over validity of the results. The results echo those of a systematic review I led about telemedicine studies several years ago [7]. One concern about this new review is that its methodology of being a review of reviews might magnify poor evidence. But someone needs to reconcile this review with the one of Buntin et al. [2].

It should be noted that another line of thought has been critical of the experimental approach to evaluation of health IT. Two recent commentaries note that these approaches cannot capture the whole picture of a health IT intervention, especially ones that occur in real-world implementations in complex settings, like states or even whole countries [8, 9]. I acknowledge these criticisms, though would argue back that we should not view these approaches as either-or. There is hopefully plenty of room for all types of disciplined evaluation of informatics, with clinical trials and similar experiments

References

1. Romano, M. and Stafford, R. (2011). Electronic health records and clinical decision support systems: impact on national ambulatory care quality. Archives of Internal Medicine, Epub ahead of print.
2. Buntin, M., Burke, M., et al. (2011). The benefits of health information technology: a review of the recent literature shows predominantly positive results. Health Affairs, 30: 464-471.
3. Goldzweig, C., Towfigh, A., et al. (2009). Costs and benefits of health information technology: new trends from the literature. Health Affairs, 28: w282-w293.
4. Chaudhry, B., Wang, J., et al. (2006). Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Annals of Internal Medicine, 144: 742-752.
5. Anonymous (2011). Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC, Institute of Medicine.
6. Black, A., Car, J., et al. (2011). The impact of eHealth on the quality and safety of health care: a systematic overview. PLoS Medicine, 8(1): e1000387.
7. Hersh, W., Hickam, D., et al. (2006). Diagnosis, access, and outcomes: update of a systematic review on telemedicine services. Journal of Telemedicine & Telecare, 12(Supp 2): 3-31.
8. Greenhalgh, T. and Russell, J. (2010). Why do evaluations of eHealth programs fail? An alternative set of guiding principles. PLoS Medicine, 7(11): e1000360.
9. Patrick, J. (2011). The validity of personal experiences in evaluating HIT. Applied Clinical Informatics, 1: 462-465.

Looking Back, Moving Forward

This week marks a year ago that I woke up (on the morning of Friday, April 2nd, to be precise) to find emails in my inbox telling me that Oregon Health & Science University (OHSU) had received our two awards from the Office of the National Coordinator for Health IT (ONC) Workforce Development Program. As most readers know, those programs are going well, and I am planning to provide my occasional updates of our efforts in the programs here in the coming weeks ahead. A succinct news report of the programs, for which I was interviewed, is available from the California Health Care Foundation (CHCF).

Another anniversary of sorts is for this blog, which has now been in existence a little over two years (since March 2, 2009, to be precise). I have enjoyed having this forum to share my thoughts about topics of interest and passion to me. I have tried to create thoughtful pieces that explore various issues, and not just brief streams of consciousness.

The year 2011 also is another anniversary year, which is the 15th year of informatics degree programs being offered by OHSU. In 1996, we opened the doors to our initial Master of Science degree. Of course, we have added a number of other degrees since then, such as our Master of Biomedical Informatics (non-thesis, professional master's), PhD, and Graduate Certificate. To celebrate the 15-year anniversary, as well as the first two groups of graduates from our ONC funding, we are planning to hold a celebration in September. The event will be open to the public and is scheduled to take place on September 9-10, 2011. All alumni, students, faculty, and friends of the program will be invited, with alumni being able to present about what they are currently doing, along with a number of other keynote speakers. (Save the date!)

Of course, we are not resting on our laurels, and are quite busy with our current work, the totality of which would be much longer than anyone would want to read. I am happy to announce that the 10x10 ("ten by ten") program continues going strong, with  a number of new offerings planned to start in the next few months. One of the offerings is a general AMIA offering but the rest demonstrate the partnerships that we have built for specific offerings. As with all 10x10 courses, the offerings include the basic on-line portion of the course and an in-person session often associated with a professional meeting.

They include:

• Regular AMIA offering aimed at all audiences, starting April 27, 2011 with in-person session at any AMIA national meeting in the next year. (Next meeting in Washington, DC, with in-person session on October 23, 2011)
• Offering focused on dietitians and the area of nutrition informatics, in partnership with the American Dietetic Association (ADA), starting April 13, 2011, with the in-person session at the ADA meeting in San Diego, CA on September 24, 2011
• Offering in Singapore in partnership with Gateway Consulting, starting May 2, 2011, with the in-person session in Singapore (!) on September 14, 2011
• Offering focused on emergency physicians, in partnership with the American College of Emergency Physicians (ACEP), starting June 29, 2011, with the in-person at the ACEP meeting in San Francisco, CA on October 14, 2011

Honorable Mention

Who should be the next National Coordinator for Health Information Technology (HIT), i.e., the Director of the Office of the National Coordinator for HIT (ONC)? My name appeared recently on a list of 24 individuals nominated as a potential replacement for the current National Coordinator, Dr. David Blumenthal, who is leaving the post next month to return to academia at Harvard University.

Well, the results of the voting are in, and I was flattered to finish fifth, capturing 5.6% of the 736 votes cast. The winner of the poll was Jessica Kahn, Technical Director for HIT for Medicaid at the Centers for Medicare and Medicaid Services. Following in a close second was Dr. Marc Chasin. Vice President and Chief Medical Informatics Officer at St. Luke's Health System in Boise, Idaho. I am delighted to report that Dr. Chasin is a former student of mine, having taken the 10x10 ("ten by ten") course.

I have to admit that I am ordinarily unswayed by magazine or Web polls that are completely unscientific and really just popularity contests. Still, I was flattered to be part of this. Perhaps, as they say in show business, any publicity is good publicity.

This all said, I don't think I will be leaving Oregon Health & Science University any time soon. I have waited my whole career for the situation I and the department I lead are currently in, seeing the maturation of our field and the resources now available to support education and research endeavors within it.

I also do not envy the person who actually replaces Dr. Blumenthal. Clearly the HITECH Act has, to use the terminology of Gartner Hype Cycle, hit its peak of expectations. Some of the programs are down in the trough of disillusionment, although I am confident that most if not all of them will eventually level off in the plateau of productivity. Given the current political state of Washington, DC, where scoring political points seems to have overtaken governing and producing value from government programs, the next National Coordinator is likely to spend a good deal of time in non-productive Congressional hearings. It's not that I don't think government bureaucrats, like everyone else, need to be held accountable, but it is unlikely the primary purpose of those hearings will be to report on the value to healthcare and economy that HITECH has wrought.

Being the optimist that I am, I am not dwelling too much on the current poisoned atmosphere in Washington, DC. I will certainly defend to anyone the productive investment that has been made the federal government in HIT. In our programs funded by educational grants, skills and leadership have been imparted on a new cadre of individuals, and the curricular materials we are producing will have a lasting impact on the primary goal of biomedical and health informatics, which is to improve human health, healthcare, biomedical research, and public health with information.

Natural Language Processing: A Dream That Won't Die … and Shouldn't

One of the longest-standing dreams of informatics, dating back to the early (i.e, 1960s) era of artificial intelligence, is the use of natural language processing (NLP) to extract data about patients from clinical narrative data (e.g., progress notes, discharge summaries, etc.) in the electronic health record (EHR). The notion that you can take the narrative language of clinicians and turn it into concrete facts that can be used for clinical decision support, clinical research, quality measurement, surveillance, etc. is immensely appealing.

Alas, that dream, at least in a generalizable way, is still a dream. You can count a number of my published papers over the years as a few among the many valiant efforts. Unfortunately, the variability (or some might say mangling, especially by physicians) of language, along with the hidden context and meaning "between the lines," makes NLP a very difficult task to program in a computer.

Some, however, have managed to succeed in focused ways. For example, generalizable decision support also never succeeded but it has been found that focused decision support works quite well and is used in EHRs daily. Likewise, there have a number of focused areas where NLP has provided useful data for clinical processes.

It is in this context that I am pleased to report on another contribution to the literature of clinical NLP, which is a paper that appeared in a recent issue of the Journal of the American Medical Informatics Association (JAMIA) and is lead-authored by a former student, Mary Stanfill [1]. I am a co-author. It is always a thrill to see a student publish a peer-reviewed paper, especially one that started as a term paper in one of my courses, advanced to a capstone project in a master's degree, and ultimately ended up in one of the leading journals in our field.

This paper also makes a valuable contribution of being a systematic review of all studies that report results of "automated coding and classification." The analysis shows that there have been many efforts performed using many methods in a variety of clinical domains, with a wide range of results. Of course, this gets to a gripe I have had with clinical NLP and related text mining researchers over the years, which is that evaluation studies have not advanced much beyond measuring the accuracy (e.g., recall, precision, sensitivity, specificity, etc.) of how well systems identify concepts in the text [2]. I would prefer to see the next step in systems being evaluated, such as how well NLP can impact the tasks it might be used for, such as quality measurement programs or facilitating clinical research studies. This would be akin to the "task-oriented" studies of information retrieval systems I performed years ago, which focused on how well searchers completed tasks using retrieval systems rather than just measuring how many relevant articles they retrieved [3].

The good news is that systems using NLP are starting to be deployed in operational clinical settings or clinical and translational research programs, and there is an ever-increasing amount of real data in electronic form for them to use. In addition to a growing number of individual studies, there are also large-scale projects of which NLP is a significant part. There include:

• Informatics for Integrating Biology and the Bedside (i2b2) - a long-standing project to facilitate the use of clinical data for genomic and clinical research. One of its activities includes a yearly challenge evaluation that allows research to compare systems and results on a common task. The i2b2 challenge has looked at automatic de-identification [4], identification of smoking status [5], recognition of obesity and co-morbidities [6], and extraction of medication information [7].
• Electronic Medical Records and Genomics (eMERGE) Network - a multi-center project focused on the use of data in EHRs to facilitate the study of how genetic variability contributes to health and disease [8]. One of the foci includes the use of NLP for extracting data from clinical narratives and integrating it with other data in the clinical record. One accomplishment of this research to date has been the ability to replicate four of seven known gene-disease associations [9].
• SHARP 4 - one of the four collaborative research centers being funded under the HITECH Program to facilitate meaningful use of EHRs, with a focus on secondary use of EHR data.

Another development is the launching this year of a medical records track in the Text Retrieval Conference (TREC) annual information retrieval challenge evaluation. The track will use de-identified records developed by Wendy Chapman and colleagues.

It is also impossible to discuss this topic without acknowledging the discussion around the IBM Watson question-answering system, which recently proved its mettle in a television game show Jeopardy match [10]. IBM has announced some research partnerships that will apply Watson to medical data. This is an interesting research area, but we will need to see real research results to back up the hype [11].

While there are still challenges for clinical NLP, I believe we are seeing a convergence of new methods coupled with growing needs to make use of the increasing volume of clinical data as well as our desire to facilitate re-use of that data for many purposes, such as clinical decision support, quality measurement and improvement, clinical research, and public health reporting and surveillance. While there may be generalizable approaches yet to be discovered, I suspect that evolution will be much like clinical decision support, which has been more successful when engineered to specific domain areas. But as we have also seen with clinical decision support, the ability to perform those specific tasks successfully will be highly valuable to healthcare.

References

1. Stanfill, M., Williams, M., et al. (2010). A systematic literature review of automated clinical coding and classification systems. Journal of the American Medical Informatics Association, 17: 646-651.
2. Hersh, W. (2005). Evaluation of biomedical text mining systems: lessons learned from information retrieval. Briefings in Bioinformatics, 6: 344-356.
3. Hersh, W., Crabtree, M., et al. (2002). Factors associated with success for searching MEDLINE and applying evidence to answer clinical questions. Journal of the American Medical Informatics Association, 9: 283-293.
4. Uzuner, O., Luo, Y., et al. (2007). Evaluating the state-of-the-art in automatic de-identification. Journal of the American Medical Informatics Association, 14: 550-563.
5. Uzuner, O., Goldstein, I., et al. (2008). Identifying patient smoking status from medical discharge records. Journal of the American Medical Informatics Association, 15: 14-24.
6. Uzuner, O. (2009). Recognizing obesity and comorbidities in sparse data. Journal of the American Medical Informatics Association, 16: 561-570.
7. Uzuner, O., Solti, I., et al. (2010). Extracting medication information from clinical text. Journal of the American Medical Informatics Association, 17: 514-518.
8. McCarty, C., Chisholm, R., et al. (2010). The eMERGE Network: a consortium of biorepositories linked to electronic medical records data for conducting genomic studies. BMC Genomics, 4(1): 13. http://www.biomedcentral.com/1755-8794/4/13.
9. Denny, J., Ritchie, M., et al. (2010). PheWAS: Demonstrating the feasibility of a phenome-wide scan to discover gene-disease associations. Bioinformatics, 26: 1205-1210.
10. Ferrucci, D., Brown, E., et al. (2010). Building Watson: an overview of the DeepQA Project. AI Magazine, 31(3): 59-79. http://www.aaai.org/ojs/index.php/aimagazine/article/view/2303.
11. Anonymous (2011). IBM to Collaborate with Nuance to Apply IBM’s "Watson" Analytics Technology to Healthcare. Armonk, NY, IBM. http://www-03.ibm.com/press/us/en/pressrelease/33726.wss.

PCAST Report: What's the Big Deal?

As anyone who works in informatics knows, there is unremitting stream of reports, white papers, blog entries, and other writings from various government agencies, non-profit organizations, consultants, research organizations, and others involved in health information technology (HIT). Some of these reports promote various points of view, including policy directions, while others present interesting ideas to read. (Some do neither!)

One recent report has garnered more attention than any in the last several months (perhaps since the release of the meaningful use rules). This is of course the recent report from the President's Council of Advisors on Science and Technology (PCAST) entitled, Realizing the Full Potential of Health Information Technology to Improve Healthcare for Americans: The Path Forward, which was released in December, 2010. This report, called the “PCAST report” by many, has gained high visibility, perhaps reflecting its origin from the White House. It has led the Office of National Coordinator for HIT (ONC) to ask its HIT Policy Committee to create a workgroup tasked with collecting and analyzing public comments and making recommendations relative to current and future ONC activities.

The report states its case by noting that current HIT systems do not meet their potential, mainly due to the lack of interoperability that results from proprietary data stores (mostly in proprietary systems) that blunt the free flow of data. This is hardly new. The report goes on to advocate what it views as the critical solution to the problem, which is the development of a “universal exchange language” (abbreviated by some though not in the report as “UEL”) based on the notion of data elements being reduced to their atomic core. It advocates that each of these core elements have metadata (“data about data”) tagging that includes the element and its value along with an identity of the patient, a patient-controlled privacy designation, and other provenance information about the element.

The report has certainly generated a great deal of discussion, with most of the major organizations involved in HIT having weighed in during the ONC comment period. The report certainly piqued my interest, since I have always held the view that data is the most critical aspect of everything we do with HIT. I agree with Dr. Blumenthal that data is the “lifeblood” of medicine, and my informatics nirvana consists of data freely flowing between systems and the appropriately authorized people to use it. In my dream world, one could be in one EHR and “Save as…” the data for loading into another EHR or some other application. The data would be so interoperable that any application would recognize documents (e.g., discharge summaries or progress notes), measurements (e.g., vital signs or lab values, and structure data (e.g., prescriptions).

So where does PCAST fit into all of this? In this posting, I will review the PCAST Report, summarize the commentary and criticisms, and give my own further analysis to get others thinking (as a good educator should!).

The report begins with the usual accolades for how HIT has the potential to transform healthcare. In addition to the usual improving clinician access to patient data and decision support, involving patients in their care, and enabling public health and clinical research, the report also notes that HIT can create new markets and jobs for technology as well as support healthcare reforms, including economic changes in the system. It lays out nine use cases that benefit patients, clinicians, public health, and clinical research.

However, the report notes, HIT has fallen short of its promise for four reasons. First, most current systems are proprietary applications not easily adapted into clinical workflow, with proprietary data formats not easily exchangeable. Data is not easily disaggregated or re-aggregated. Second, most healthcare organizations focus on the internal value of EHRs and have no incentive for secondary or external uses of their data for patients, other healthcare organizations, public health, or research. A third reason is that patients have concerns about the privacy and security of information of their data. Finally, the report notes that HIT has been largely focused toward administrative functions and not on improving healthcare.

The UEL will require a common infrastructure for locating and assembling data elements of a patient’s records via secure data element access services (DEAS). Data would remain local, and DEAS would be distributed, intercommunicating, and interoperable. A single appropriately authorized query would be able to locate and assemble a patient’s record across multiple DEAS.

The essential core of the report is Chapters 4-5 (pp. 39-52), which deal with the core of the technology and privacy. Chapter 4 asserts that the approach of applications as “services” does not scale up. (Though the authors seem to violate the notion of separating the application and the data.) It is argued that a better approach for healthcare data is the UEL, which is coded in (eXtensible Mark-up Language) XML and tagged with three metadata elements (in addition to the data and its value):

• Identifying information about patient – including information enabling location of the data (not necessarily a universal identifier)
• Privacy protection information – who may access the data, for what purposes, and either in an identified or de-identified state
• Provenance of data – date, time, equipment used to collect data, personnel who collected it, etc.

The UEL would aim for semantic interoperability. While adherence to specific controlled terminology sets would not be required, it would be strongly encouraged. This would allow over time for data to truly be represented in a universal way.

DEAS activities would include “crawling, indexing, security, identity, authentication, authorization, and privacy.” Queries would be issued against all DEAS on the Internet, and results could be re-constituted into a complete picture of patient. Governments, healthcare organizations, and others would operate the DEAS. In some way, this process would act like Web search engines, although they would need to have very high recall and precision to insure all the appropriate data was retrieved, while incorrect data was not. In conclusion, the chapter claims the UEL is extensible, extractable by middleware, and will lead to innovative uses and applications.

Chapter 5 lays out the privacy and security aspects of the UEL and DEAS. In essence, each and every data element would have a patient-controlled privacy attribute, allowing access to the element and its use in identified or de-identified scenarios. All data would be encrypted, which would not only protect security, but also insert a mechanism to audit access.

A number of leading HIT organizations took the opportunity of the ONC comment period to state their positions. While all applauded the raising of awareness of issues related to data and its interoperability, they also raised a number of criticisms. Many advocated that the PCAST Report serve as a broader vision rather than holding concrete solutions.

The comments about the report can be summarized as follows:

1. The constellation of current standards, as imperfect as they are, meet many of the data-related goals laid out by the report.
2. Many of the ideas of the report, while interesting and worthy of further research, are untested. Having them be the drivers of Stage 2 of meaningful use would be a substantial change in direction and put the larger HITECH Program at risk.
3. Clinical data has context, and reducing its entirety to atomic elements has the potential to lose that context. Re-constituting it may not be possible if that context is lost.
4. Much of the context in clinical data requires that records be more document-centric or at least structured in groups of elements. Disaggregating documents could lose the context they provide.
5. While everyone agrees that structured data is most desirable, some data in healthcare is too nuanced, and unstructured text is required to describe it.
6. While industry-wide standards are important, no industry with data as complex as healthcare has tried an approach like this.
7. The notion of setting privacy at the individual element is highly problematic. Allowing the patient to choose which elements can be seen or not seen by clinicians, researchers, or others could introduce many unintended consequences. It would be preferable for the patient to specify general privacy policies that are then referenced by the data elements.
8. Patients’ views of privacy may change over time, as diseases change and their own disease course changes.
9. Search engines are imperfect. The DEAS would need to operate at high levels of recall and precision that are unprecedented for Internet-based search mechanisms.
10. While the report correctly notes that current HIE efforts are struggling, it ignores that major reasons for this, which have more to do with the lack of a business model for HIE than anything about the technology.

A number of specific comments from these organizations are poignant:

• American Hospital Association (AHA) - The report should set a broader vision rather than focus on concrete solutions. Setting privacy at element level could fracture the patient record. Tagging each piece of data could be costly and inefficient. DEAS are likely to face same challenges as HIEs, with lack of a business model. ONC should change policy direction for Stage 2 of meaningful use only with great caution.
• Radiological Society of North America (RSNA)/American College of Radiology (ACR) - Echoed many of the same comments and noted we need a uniform method to manage patient identity.
• Integrating the Healthcare Enterprise (IHE) USA - Noted that the current IHE profiles cover most of the functionality required for the 9 use cases.
• Healthcare Information and Management Systems Society (HIMSS) -Privacy is contextual and changing, especially as diseases change and information about them becomes less sensitive. Encryption of the data elements provides security and an audit trail but can adversely impact workflow. The objectives of the report might not be possible without a universal patient identifier. By atomizing data, we run risk of data becoming dissociated and not being able to detect errors, so any grouping in the source data should be maintained. Metadata tagging should be virtual and not physical. Tags should be referenced and not attached, since some (e.g., privacy) might change over time. Data elements separated from documents and records potentially robs them of their context.
• HIMSS Electronic Health Record Association (EHRA) -It is better to tag data on document or record level. The privacy approach is potentially unworkable. A large-scale effort of this approach is untested.
• AMIA - Chapter 4 provides general ideas but no details nor references. There is no evidence that this approach will lead to improved care. The report was for the most part silent about other federal agencies, especially the National Library of Medicine, which has great expertise in some aspects of the proposed approach, especially related to terminology development and usage. The report underestimates the complexity of modeling the domain of medicine. It ignores past failed efforts along similar lines, such as the caBIG caDSR project. The DEAS may not be scalable or practical. ONC should not deviate from already tight timeline of Stage 2 of meaningful use. We can learn lessons from the slow adoption of HL7 Version 3, which is not suited to efficient description of task information models. There is too much focus on healthcare and not enough on health.

Other organizations that weighed included:

• HL7
• Federation of American Hospitals (FAH)
• Center for Democracy in Technology
• Project HealthDesign
• Markle Foundation
• Clinical Groupware Collaborative
• Faster Cures
• Society for Participatory Medicine
  

And of course, a number of bloggers had things to say. As always, John Halamka provided great early summaries of the report and the deliberations of the ONC HIT Policy Committee:
http://geekdoctor.blogspot.com/2010/12/spirit-of-pcast.html
http://geekdoctor.blogspot.com/2011/01/primer-on-xml-rdf-json-and-metadata.html
http://geekdoctor.blogspot.com/2011/01/example-for-pcast.html
http://geekdoctor.blogspot.com/2011/01/general-principles-of-universal.html

In a widely cited posting, Wes Rishel noted some critical points: Information flow for patient care occurs at the level of documents. Taking elements out of larger context can lose context. PCAST data elements are actually molecules, not atoms. There are plenty of molecule definitions, these should be used. Another well-known blogger, Keith Boone, added that a good deal of what the report hopes to accomplish can be done with existing standards.

What will be the impact of the PCAST report? We will find out for sure in April when the ONC releases its analysis and plans for incorporating the report’s ideas and proposals. If nothing else, the report has led to increased discussion about the importance of data interoperability, which even its critics applaud. My hope is that there is at least an acceleration toward the vision of interoperable data that most in informatics share.

Some Supplemental Information from the PCAST Report

The nine use cases were lumped into three categories based upon to whom they provided value.

Value to patients:

• Patient on warfarin wanting to know if it is safe to take an NSAID drug for an injury.
• Woman with lung mass newly discovered in a community hospital referred to a large academic medical center.

Value to clinicians:

• Internist developing primary care medical home.
• Small practice with clinicians sharing records and communicating with patients via email.
• Cardiology clinics in a part of country collaborating to improve care for patients with recent myocardial infarction.
• Family physician embedding alerts in practice to improve preventive care.

Value to research and public health:

• Physician caring for a patient enrolled in a national clinical trial.
• FDA carrying out post-marketing surveillance of adverse reactions to drugs.
• Communities or states measuring improvement toward health goals.

The final published conclusions of the report were:

1. HHS and ONC have laid a foundation for progress under meaningful use and HITECH.
2. Achievement of goals for healthcare involves accelerated progress toward robust health information exchange.
3. Effort should now focus on development of a universal exchange language that enables data to be shared and re-assembled across institutions, subject to strong privacy safeguards based on patient privacy preferences.
4. Creating these requirements is technically feasible.
5. ONC should move rapidly to develop these capabilities for stages 2 and 3 of meaningful use.
6. CMS should modernize and restructure its IT platforms to serve as a driver for progress in health IT.

Follow-up from HIMSS

This year's Healthcare Information Management and Systems Society (HIMSS) Annual Conference in Orlando, Florida was interesting and enjoyable as always. Clearly the focus was on healthcare organizations and vendors achieving meaningful use of electronic health records (EHRs) . However, there was also plenty of other buzz on topics such as health information exchange, the impact of the departure of Dr. Blumenthal, and changes going on in Washington, DC.

There was also, of course, much talk about health IT workforce issues and the ONC Workforce Development Program. I was video-interviewed on aspects of the topic by:

• Joseph Conn of Modern Healthcare, who produced a six and a half minute video on workforce issues and educational programs
• Mary Stevens of CMIO Magazine, who produced a three and a half minute video on education and career opportunities for physicians, especially those aspire to be CMIOs.

Accolades for the Informatics Professor and His Blog

I have had the opportunity to publish and speak in some high-profile venues lately, and my blog and my book have received some honors. For the sake of the reader, however, I will roll all of my "bragging" into just this single post.

Let me start with publishing, where I have been invited to make some commentaries on various aspects of informatics. One of these was a contribution to a series of commentaries on the HITECH program in the prestigious journal Nature. (Subscription required for HTML and PDF versions.)

Another was an invitation to write an editorial accompanying a study on the development of registries in chronic kidney disease in the journal, Clinical Journal of the American Society for Nephrology (subscription required).

I have also be invited to talk on panels at a couple of high-profile meetings. One talk was on a panel at the AcademyHealth National Health Policy Conference (February 7-8, Washington, DC). The panel was focused on workforce issues, with speakers addressing different aspects of the healthcare workforce issue. Naturally I was invited to speak on informatics issues, describing the ONC Health IT Workforce Program as well as informatics competencies for modern healthcare professionals (slides).

Another talk is on a panel at this week's HIMSS Annual Conference on the ONC Health IT Workforce Program. I will speak about the Health IT Curriculum Development Centers Program (slides). OHSU serves not only as one of the five Curriculum Development Centers but also the National Training & Dissemination Center tasked with disseminating the curricular materials via a Web site and training community college faculty in their use.

I am also pleased to report that the Informatics Professor blog has been recognized by three sites that rate blogs:

• Top Professors of Math & Science Blogs
• Forty-Six Health Care Blogs for Professionals
• 50 Resources for Students Attending Online Health Information Schools

Finally, I am honored to have my book reviewed by one of my favorite blogs, Keith Boone's Healthcare Standards blog. It turns out that Keith did some work in search earlier in his career and has some poignant things to say about my book (and my blog). I look forward to his book on Clinical Document Architecture due out this spring.

Welcome to Oregon, President Obama and Mr. Otellini

Today I will have the opportunity to attend President Obama's visit to Intel Corp. in Portland, Oregon. While I will just be among the crowd, I would love the opportunity to talk with both of them, and will post my comments of what I would say in this blog entry.

President Obama, I want to thank you for the opportunity afforded by the HITECH Act in your stimulus legislation. I wish we could have you over to visit Oregon Health & Science University (OHSU), where we have put the $5.8 million awarded to us under the HITECH Act to good use. It is an investment in human capital that is building the workforce of future professionals who will lead the "meaningful use" of electronic health records (EHRs) to improve the quality and safety of healthcare. On time and on budget, we are not only one of nine universities training these professionals from around the country, but also one of five universities developing curricular materials for shorter-term community college programs. In the latter program, we also serve as the National Training & Development Center, training and supporting community college faculty in use of the materials and disseminating them via a secure Web site. We have also enhanced the scope of the program by building a curriculum around the VistA EHR system from the Veteran's Administration. This system is another exemplary federal project that has drastically improved healthcare delivery by the VA, and now will provide additional value in training future health information technology professionals in using and configuring EHRs.

To Intel Corp., namely Mr. Paul Otellini, CEO, I would love the opportunity to show off the work being done by a local university, OHSU, in your company's own backyard here in Oregon. I would relish the opportunity to work with your company more to increase economic development and create high-skill, high-paying jobs in Oregon. I have written in this blog, a local tech blog, and even our local newspaper, the Oregonian, about the potential that exists in Oregon for synergy among companies, an innovative healthcare system, and a world class academic program in biomedical informatics at OHSU to create an economic cluster around health information technology. Investing in students, research, and companies could pay off well for our region.

Postscript: I had an enjoyable time visiting Intel and hearing President Obama speak. I managed to get a front row seat and was able to shake the President's hand afterwards. Of course I took pictures and posted some of them on Facebook. I still would love the opportunity to address either the President or Mr. Otellini about the issues raised above.

HITECH: Improving Healthcare Through Data and Action

Every now and then, I am asked to give an overview of the Health Information Technology for Economic and Clinical Health (HITECH) Act of the American Recovery and Reinvestment Act (ARRA, also known as the “economic stimulus bill”). The centerpiece of HITECH is a plan to vastly expand the adoption and “meaningful use” of electronic health records (EHRs) [1], based on a growing body of research demonstrating that EHRs, especially when combined with clinical decision support (CDS), can improve the quality, safety, and coordination of healthcare [2, 3]. Similar to other areas related to technology and/or healthcare, the US has become a laggard in the adoption of EHRs, falling behind most other developed countries [4].

HITECH provides up to $27 billion for eligible professionals and hospitals to receive incentives for achieving the meaningful use of EHRs [5]. Meaningful use is a critical concept. The goal of HITECH is not just to put computers into physician offices and on hospital wards, but rather to use them toward five goals for the US healthcare system: improve quality, safety and efficiency; engage patients in their care; increase coordination of care; improve the health status of the population; and ensure privacy and security. As such, every criterion in meaningful use (e.g., drug-drug interaction checking) must tie back to a healthcare goal (e.g., improve quality, safety and efficiency).

Government funds for HITECH incentives will be distributed through the public Medicare and Medicaid reimbursement systems. Depending on choice of funding through Medicare or Medicaid, eligible professionals can receive $44,000-$63,000, while eligible hospitals can receive $2-9 million between 2011 and 2018. The main purpose of these incentive funds is to cover the costs of investment in EHR systems. It is anticipated that further costs will become part of the "costs of doing business" for healthcare.

The HITECH legislation recognizes that incentives alone will not be enough to achieve all the goals of meaningful use. As such, HITECH allocates an additional $2 billion for various human and organizational infrastructure elements to attain its mandates. A critical portion of this infrastructure is the ability to achieve health information exchange (HIE), which is the secure flow of data to wherever it is needed for patient care, including across traditional business and other boundaries in the healthcare system [6]. About $547 million is allocated to states for HIE development.

Another critical piece of the infrastructure is the provision of technical support to achieve meaningful use. This is done with the allocation of about $677 million to 62 regional extension centers that are providing a variety of forms of assistance, mainly to small primary care practices [7].

An additional portion of the required infrastructure is a competent professional workforce to develop, implement, and train users of EHR and related systems. It has been estimated that the HITECH agenda will require an additional 50,000 professionals trained in fields such as biomedical informatics and health information management [8]. About $118 million has been allocated for both short-term training programs in community colleges as well as longer programs mostly at the graduate level in universities. My institution, Oregon Health & Science Univeristy, is playing a major role in this program.

The HITECH legislation also recognizes that additional research and development is required. As such, $60 million has been allocated to establish four collaborative research centers focusing on the topics of security and health information technology, patient-centered cognitive support, health care application and network design, and secondary use of EHR information. A related funding initiative is the Beacon Communities Program, which has funded about $250 million for 17 advanced demonstration projects “shine the light” forward.

Just as meaningful use connotes that EHR adoption is not just about installing computer technology in clinical settings, there are related initiatives in the United States that will synergize with the substantial HITECH investment. One initiative from the Institute of Medicine aims to develop the “learning health care system” that learns from the growing volume of captured data what does and does not work in healthcare [9]. This is closely related to the growing push for “comparative effectiveness research” that aims to compare tests, treatments, and other medical activities in head-to-head studies carried out in real-world settings [10]. This infrastructure will also likely contribute to the growing push for translational research, as exemplified by funding for the Clinical & Translational Science Award (CTSA) program of the National Institutes of Health [11].

Taken collectively, all these programs from HITECH to ACA, the learning healthcare system, and CTSA provide a vision of a new healthcare system that learns from its successes and changes based on its mistakes. This vision uses data as the critical enabler of coordinating, measuring, and researching care. HITECH is indeed a grand experiment, and it is likely be that some elements of this experiment will succeed whereas others fail. But in the end, the healthcare system should benefit this unprecedented investment in information systems, human capital, and goals for improving health.

References

1. Blumenthal D, Launching HITECH. New England Journal of Medicine, 2010. 362: 382-385.
2. Garg AX, Adhikari NKJ, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, et al., Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. Journal of the American Medical Association, 2005. 293: 1223-1238.
3. Goldzweig CL, Towfigh A, Maglione M, and Shekelle PG, Costs and benefits of health information technology: new trends from the literature. Health Affairs, 2009. 28: w282-w293.
4. Schoen C, Osborn R, Doty MM, Squires D, Peugh J, and Applebaum S, A survey of primary care physicians in eleven countries, 2009: perspectives on care, costs, and experiences. Health Affairs, 2009. 28: w1171-1183.
5. Blumenthal D and Tavenner M, The “meaningful use” regulation for electronic health records. New England Journal of Medicine, 2010. 363: 501-504.
6. Vest JR and Gamm LD, Health information exchange: persistent challenges and new strategies. Journal of the American Medical Informatics Association, 2010. 17: 288-294.
7. Maxson E, Jain S, Kendall M, Mostashari F, and Blumenthal D, The regional extension center program: helping physicians meaningfully use health information technology. Annals of Internal Medicine, 2010. 153: 666-670.
8. Hersh W, The health information technology workforce: estimations of demands and a framework for requirements. Applied Clinical Informatics, 2010. 1: 197-212.
9. Eden J, Wheatley B, McNeil B, and Sox H, eds. Knowing What Works in Health Care: A Roadmap for the Nation. 2008, National Academies Press: Washington, DC.
10. Murray RK and McElwee NE, Comparative effectiveness research: critically intertwined with health care reform and the future of biomedical innovation. Archives of Internal Medicine, 2010. 170: 596-599.
11. Zerhouni EA, Translational research: moving discovery to practice. Clinical Pharmacology and Therapeutics, 2007. 81: 126-128.

Electronic Health Records Do Not Impact the Quality of Healthcare?

The headlines have blared the news this week that a new study published in Archives of Internal Medicine showed that electronic health records (EHRs) in the ambulatory setting do not appear to lead to higher quality patient care [1]. This in turn has led many leading news organizations to have stories with headlines such as, Stanford researchers find EHRs don't boost care quality.

For those of us who work in informatics, this is a pretty serious finding. As responsible scientists and citizens, we cannot ignore negative results about the work we do. However, we also have an obligation to place this work in the larger context of all research on the relationship between health information technology (HIT) and quality of medical care.

Like almost all science that gets reported in the general media, there is more to this study than what is described in the headlines and news reports. The study was published in a prestigious medical journal by two Stanford researchers. The implementation of the research methods they used appears to be sound. There is no reason to believe that the results obtained do not derive from the methods used.

However, there are serious limitations to this type of study and to the data resources used to answer the researchers' question, which was whether ambulatory EHRs that include clinical decision support (CDS) lead to improved quality of medical care delivered. While I do believe this study has a place in the evidence base of HIT, it suffers from limitations that are inherent in studies like this that that are observational, correlational, and retrospective. This study used a data source collected for other purposes, the National Ambulatory Medical Care Survey, and compared physicians who were identified users of CDS with those who were not to see if there were differences in the quality of care they provided based on 20 process quality measures. The results found there were no differences between the groups, i.e., those using EHRs and CDS did not deliver higher quality care than those not using them.

Before delving into the details, it is also worth noting that this study is not the first to apply this methodology. Within the last couple months, two other studies have used a similar approach to assess associations between quality measures and hospital EHR adoption [2] and computerized provider order entry [3], giving mixed results, i.e., some measures showing benefit and others not. In addition, Archives of Internal Medicine published another study using this sort of approach in 2009 showing that hospital notes, test results, order entry, and decision support were variably associated with improved patient outcomes and reduced costs [4]. (It was surprising to see the latter not referenced in the article.) If we were to take all of these studies as definitive, then we might conclude that EHRs usage in hospitals improves quality of care, even if EHR usage does not in ambulatory settings.

But whether the results are favorable or not, it is important to understand some serious limitations in these types of studies and this one in particular. A first limitation is that the study looks at correlation, which does not mean causality. This was an observational and not an experimental study. The data used for the study was not collected for the purposes of assessing the quality of care by EHRs. As with any correlational study, there may be confounders that cause the correlation or lack of it. As we know from evidence-based medicine, the best study design to assess causality is an experimental randomized controlled trial. Indeed, such studies have been done and many have found that EHRs do lead to improvements in quality of care. There have been several systematic reviews of such studies noting that while some of the studies suffer from methodologic limitations, others are well-designed and do demonstrate positive value for various aspects of EHR and CDS [5-7]. There is a continuing stream of such studies and two have been published in the last couple months. One showed that an EHR with targeted CDS led to increased improvements in glucose control in diabetics [8], while another found that a real-time alert cut inappropriate use of D-dimer testing by 70% [9]. Not all such studies demonstrate positive results, but enough to do to show that there is value in the well-informed use of HIT.

A second limitation of this study is the quality measures used. Quality measures are of two general types, process and outcome. Process measures look at what was done, such as ordering a certain test or prescribing a specific treatment. Outcome measures look at the actual clinical outcomes of the patient, e.g., whether there was a reduction of mortality, complications, or cost. It is a fair criticism of the current state of the healthcare quality movement that most measures used (including those in the meaningful use criteria) are process measures that may or may not result in improved patient outcomes.

A third limitation of this study is that we do not know whether the physicians using EHRs and CDS had decision support in place to impact the specific quality measures that the researchers studied. While the quality measures are important process measures for physicians to adhere to, they may not be amenable to CDS generally or the CDS used in these systems.

A fourth limitation is that the study assesses episodes of care and not longitudinal care over time. This may not portray an accurate picture of a physician's practice.

A fifth limitation is that the data analyzed was collected in 2005-2007. While EHRs and CDS were available at that time, they were less widely used and less mature at that time.

Finally, we have no idea how well trained these physicians were at using the CDS that they had. We know that success with HIT is based on many factors that go well beyond the technology itself, such as proper implementation and training. Well-designed research must address these factors too.

There are also some other limitations to the study that are discussed in an accompanying editorial that unfortunately few people, especially those who get news of the study from news reports as opposed to the journal itself, will read. The editorial writers appropriately point out that other studies, including experimental ones, have shown value for HIT interventions.

The results of this study are a legitimate addition to the evidence base of informatics and cannot be dismissed out of hand. However, these findings must take their place in the proper context of all research on HIT. If nothing else, this study highlights the need for more and better research to truly identify where HIT helps, has no impact, or outright harms patients.

References
1. Romano MJ and Stafford RS, Electronic health records and clinical decision support systems: impact on national ambulatory care quality. Archives of Internal Medicine, 2011: Epub ahead of print.
2. Jones SS, Adams JL, Schneider EC, Ringel JS, and McGlynn EA, Electronic health record adoption and quality improvement in US hospitals. American Journal of Managed Care, 2010. 16: SP64-SP72.
3. Kazley AS and Diana ML, Hospital computerized provider order entry adoption and quality: an examination of the United States. Health Care Management Review, 2011. 36: 86-94.
4. Amarasingham R, Plantinga L, Diener-West M, Gaskin DJ, and Powe NR, Clinical information technologies and inpatient outcomes: a multiple hospital study. Archives of Internal Medicine, 2009. 169: 108-114.
5. Garg AX, Adhikari NKJ, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, et al., Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. Journal of the American Medical Association, 2005. 293: 1223-1238.
6. Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E, et al., Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Annals of Internal Medicine, 2006. 144: 742-752.
7. Goldzweig CL, Towfigh A, Maglione M, and Shekelle PG, Costs and benefits of health information technology: new trends from the literature. Health Affairs, 2009. 28: w282-w293.
8. O'Connor PJ, Sperl-Hillen JM, Rush WA, Johnson PE, Amundson GH, Asche SE, et al., Impact of electronic health record clinical decision support on diabetes care: a randomized trial. Annals of Family Medicine, 2011. 9: 12-21.
9. Palen TE, Price DW, Snyder AJ, and Shetterly SM, Computerized alert reduced D-dimer testing in the elderly. American Journal of Managed Care, 2010. 16: e267-e275.

Reflections at the End of Another Amazing Year for Informatics

Last year, in wrapping up the first year of the Informatics Professor blog, I marveled at how amazing the year of 2009 had been. I noted that the year started with both uncertainty and hope; the former fueled by the recession and the precarious financial state of Oregon Health & Science University (OHSU) due to that recession and the latter driven by the excitement of the election of President Barack Obama and (at least for me) the hope for real change. By the end of 2009, it was clear that profound change had indeed occurred, if not generally then at least in the biomedical and health informatics field.

The hope and change, of course, were driven by the HITECH program with the president's economic stimulus package. At the end of 2009, the path forward was clear: health information technology would be driven by the concept of "meaningful use," and the part nearest and dearest to my heart, education and training, would be driven by the ONC Workforce Development Program, which itself was driven by Section 3016 of the HITECH Act that I played a role in influencing.

I spent the latter days of 2009 and early part of 2010 writing proposals, in particular for the curriculum development program and the university-based training program. With the Funding Opportunity Announcements (FOAs) for these and other programs, such as Beacon, SHARP, and regional extension centers, released in December and due in January, many in the informatics field lamented that ONC stood for the "Office of No Christmas." I spent a good part of my winter break last year working on these proposals. The only enjoyable aspect of the process was that they allowed us to envision how we could implement the educational programs we always dreamed of if we ever had the money, which now it looked like we did.

The most harrowing part of the year was the time between the submission of the proposals and receiving word about funding. As well-positioned as we were to receive these competitively awarded proposals, there was an undercurrent of fear that perhaps we forgot to address some required aspect of the program or that some reviewer felt we had taken the wrong approach. In all honesty, it would have been quite an embarrassment to not be selected for funding, since OHSU's program laid the groundwork for some of the thinking that had emerged surrounding health IT workforce development.

All the agony came to an end on Friday, April 2nd, when I awoke in the morning to find out that both OHSU proposals had been funded. For the curriculum development project, we were not only funded as one of the five curriculum development centers, but also chosen as the lead National Training and Dissemination Center (NTDC). For the university-based training program, we were one of nine programs selected for funding tuition assistance in our graduate program. A common quip in academia is that the downside to getting grants funded is that you then have to do the work. However, this was literally a dream come true. Between both grants, we were funded for $5.8 million to do what we always envisioned we could do if we had the funding. While the short-term emphasis of the funding (due to their being stimulus funds) required us to make some decisions we might otherwise not make, it was still a great position in which to be.

Also on the second to last day of 2009, the preliminary meaningful use rules were released. These were followed by a 60-day comment period, modification of the rules, and the release of the final rules on July 13th. I happened to be in a hotel room in Singapore (10 pm local time, 10 am Eastern time) when listening to their unveiling. While everyone had qualms with this criteria or that criteria, I believe that the majority of people were content with the approach to meaningful use taken by ONC.

With our own projects, we hit the ground running. Out of the gate, the curriculum development project required the most work up front. After a two and a half day workshop in Washington, DC the second week of the grant, we began our long quest that would result in the first version of the curriculum being developed and handed off to the community colleges by the end of October. Being the NTDC, OHSU also had to organize a training event for community college faculty in August and launch a Web site for dissemination of the materials around that time, both of which we did. We even added an aspect to the project of creating an educational version of the VA VistA electronic health record system.

The university-based training grant project was a little slower to get started, but not by much. With funding for 135 Graduate Certificate and 13 master's degree students over three years, our plan was to use the funding mainly as a form of tuition assistance for new students entering the field. We started providing support for students in the summer academic quarter and really ramped up in the fall. The main regret is that we have received two to three times as many qualified applicants as we having funding to accept. A decent proportion of those individuals have enrolled as self-funded students.

While a good proportion of my year was spent around these ONC initiatives, there were other achievements as well. Due to ONC and other funding, the Department of Medical Informatics & Clinical Epidemiology catapulted to second among the 25 departments at OHSU in external funding. We have many other initiatives in comparative effectiveness research, bioinformatics, and related areas. The big challenge for the department in 2011 and beyond is how to consolidate and build upon the success of the stimulus-era funding. I am confident we will find ways to do this, as the need for our disciplines to advance healthcare, personal health, and biomedical research will not diminish even as the federal budget tightens.

The coming year will also be an interesting one for the informatics world. How many eligible professionals and eligible hospitals will achieve meaningful use? What unforeseen bumps in the road will emerge? How will healthcare reform impact the use of health information technology? What will happen to healthcare reform itself? One thing is certain: we will live through exciting times!

I have now been writing this blog for almost two years. I have been pleased to have this type of forum to share my views on various aspects of my work. I am also pleased that others have noticed, not only the 129 people who follow the blog, but also winning awards like being on the list for the 2010 Top Math & Science Professor Blogs Award.

I plan to keep running the blog pretty much like I have been, with a fewer number of more substantive posts than the stream of consciousness approach used by many other blogs. I do hope to branch out a little bit more this coming year beyond workforce and education, as I occasionally did this year.

The Comfort of Connectivity

My family, friends, and colleagues believe I spend way too much time keeping up with my email and related work activities, even when I am on vacation, as I am now. They are probably right, as I type this while on vacation in lovely Oaxaca, Mexico.

Maybe it is because I remember the days when email and Internet connectivity from afar were hit or miss. Now, however, I have to admit that I marvel at the ease of accessing Wi-Fi and even my Verizon smartphone (phone, texting, and Internet on my Droid) from this lovely city that is not exactly at the forefront of technology. I am staying in a mid-range apartment, which has Wi-Fi, as does the Instituto Cultural de Oaxaca, where I am studying Spanish for a couple weeks. (The Droid is a wonderful helper for learning Spanish, as I am using two apps for Spanish-English dictionaries.) Since everything on my Droid works here, I am even able to set up a 3G mobile hot spot in a pinch!

Many eating, coffee, and other establishments have Wi-Fi as well, even the Parque El Llano a couple blocks from our apartment. My Droid has even worked in most of the small villages outside of Oaxaca. Perhaps even more amazing was that some homes in these poor villages actually have broadband Internet.

I truly am trying to take a vacation and only responding to critical emails. I have to admit there is a certain comfort to know that my connectivity is there, even if I am trying to minimize its use. I am reading my emails if for no other reason to not have thousands awaiting my return from this two and a half week vacation.

Of course, my physical and virtual lives are so merged that it would be very difficult to not use my computer and access the Internet, even when on vacation. I certainly enjoy taking pictures with my digital camera and sharing them. I also do a good deal of my news reading these days on-line. And of course there are my many friends and others on Facebook with whom I enjoy interacting. In addition, figuring out the details of visiting tourist sites, restaurants, and other places is greatly facilitated when one has Internet access. So it would be truly difficult if not impossible to completely unplug.

There is literally no place on the planet where the Internet is not accessible these days. In the past few years, I have connected from places such as Zimbabwe and Cuba. While ubiquitous global connectivity has some drawbacks, I firmly believe it is positive overall, and the ease of communication and sharing can foster better relations among peoples of the world.

10x10 ("Ten by Ten") at the End of 2010 and Beyond

With the end of 2010 approaching, I am asked with increasing frequency whether we met the goals set out by the 10x10 ("ten by ten") program, which was launched in 2005 with the goal of training 10,000 healthcare professionals in informatics by the year 2010. Now that 2010 is coming to an end, how did we do?

I can say that the program has been an unqualified success. The OHSU 10x10 offerings trained nearly 1000 (999, to be precise) people, with another eight universities training an additional 258 more, for a total of 1257 from 2005-2010. Many of those of completing the program have enhanced their current careers. From the OHSU courses, about 15% pursued additional training in the field. While our numbers did not add up to 10,000, there was clearly value for those who completed the course. The program also helped expand educational capacity in the field generally and highlighted the need that led to legislation such as Section 3016 of the American Recovery and Reinvestment Act (ARRA) and the resulting ONC Workforce Development Program.

The 10x10 courses are offered on-line, with an in-person session at the end that brings participants together face to face. The amount of material in each course is roughly comparable to an introductory three-credit graduate-level course, as shown in the syllabus from the OHSU course. In a demonstration that the Internet knows no boundaries, the course has attracted participants from all corners of the globe, such as Argentina, Hong Kong, Singapore, Israel, Pakistan, South Korea, Saudi Arabia, China, India, and Nigeria. The enthusiasm from Latin America led a group from Hospital Italiano of Buenos Aires to translate the course into Spanish and offer it across Latin America. About 500 individuals have completed this version of the course from a number of Spanish-speaking countries. Another version of the OHSU course has been offered in Singapore four times, with the in-person session held in Singapore.

We absolutely plan to continue the 10x10 program beyond the end of 2010. Two more OHSU offerings started in late 2010, along with a few more from other universities. There are no plans whatsoever to end the program, whose need continues to be demonstrated as increasing numbers of healthcare professionals and hospitals seek to achieve "meaningful use" of electronic health records. Of course, biomedical informatics is about more than meaningful use and EHRs, as demonstrated in the course syllabus.

AMIA has already changed the tag line of the program from "10,000 Trained by 2010" to "Training Next-Generation Informatics Leaders." Maybe we should just say that 10x10 now the program that aims to train 10,000 individuals in biomedical and health informatics without giving a specific deadline. Clearly the need remains.

The end of 2010 is also a time to reflect on how we arrived here. In 2005, Dr. Charles Safran, who was then President of the American Medical Informatics Association (AMIA), began taking an interest in the informatics capacity of healthcare organizations. In a letter to the editor of JAMA, he stated that each hospital in the US should have at least one physician and one nurse trained in informatics. Meanwhile, AMIA was looking to beef up its e-learning offering, but found new development of content would be prohibitively expensive. At the same time, I had already been offering the introductory course in the OHSU Biomedical Informatics Graduate Program on-line for some time. It was apparent that we could repackage the course relatively easily. Building on Charlie's call, I coined the name 10x10, aiming to train 10,000 people within five years, by 2010.

I have thoroughly enjoyed developing and teaching the 10x10 course. It has been personally gratifying to meet so many people who took the course and found it of value. I am delighted that some colleagues from Argentina translated the course to Spanish, as noted above. The course name even made its way into legislation in a bill that passed the US House of Representatives (though not the US Senate), the 10,000 Trained by 2010 Act introduced by Congressman David Wu (D-OR). A demo version is available for those who want to take a look.

Some have asked why the Chair of a department would enjoy teaching the introductory course so much. I take great satisfaction in providing people their first introduction to the field of biomedical and health informatics. I enjoy the give and take with students, including those who challenge me. The 10x10 course and my other educational accomplishments make it clear that these activities are my passion and calling in life.

The Emergence of the Informatics Practitioner

There have been many changes in the biomedical and health informatics field since its inception in the 1960s and even since my entry into it in the late 1980s. Some of these changes have been due to changes in technology, e.g., from the teletype and punch card computers of the 1960s to the advent of personal computers in the 1980s to the current era of high-powered computers and smaller devices connected to the ubiquitous global Internet. Other changes have come from scientific maturation of the field, such as a better understanding of the proper role for computerized clinical decision support and the emergence of enabling technologies from genomics and related areas.

Another area where profound change has occurred is in the professional work of informatics. When the field started to develop in the 1960s, and even when I assumed my first faculty position in the early 1990s, most who worked in informatics thought of themselves as researchers. The primary work of academic informatics departments was research and development. Most who were trained in the field obtained fellowships and/or advanced degrees. While many academic informaticians took on some operational roles in their institutions, the focus of that work was mainly implementing novel cutting-edge technology. Research, meanwhile, focused on developing new systems, models, and algorithms to meet what we thought were the needs of clinicians, scientists, consumers, and others.

Over the last decade, many changes have occurred. One of the biggest of these changes is the emergence of the informatics practitioner (or professional). Now that the use of information technology (IT) has become a routine (if mission-critical) activity of healthcare and other health-related organizations, there is growing recognition of the need for skilled individuals who understand both the technology and its use in a given underlying health domain. These professionals need not be highly technical, though they must be facile with IT and, perhaps more importantly, savvy with the management and analysis of information.

The jobs of informatics practitioners are diverse. These individuals may undertake tasks such as extracting data from "dirty" data sources (such as clinical records) for quality measurement and improvement. They may serve as champions or implementers for information systems to meet the needs of these organizations. They might maintain large bioinformatics databases or use them to analyze the data of researchers with whom they collaborate. At the top end of organizations, the chief information officer (CIO) or chief medical information officer (CMIO) increasingly provide key strategic leadership around information systems and use of the data within them.

A number of academic informatics faculty who grew up in the earlier era have not recognized the change. I have to admit that I realized it earlier than most mainly because of the demands from students in our nascent educational programs asking to learn more about how to implement systems than do research. Many academic leaders still have difficulty discerning between the differences in training researchers and practitioners.

I do not, however, see the emergence of informatics practitioners or educational programs designed for them as being at odds with the research mission of academic informatics departments. In fact, I view it as complementary. All mature fields, certainly those in the health professions such as a Department of Medicine, have both practitioners as well as researchers and educators. The researchers discover new knowledge and techniques while the educators disseminate it to the practitioners. All professions have academic departments whose missions entail both research and education. I see this starting to occur in informatics programs around the world and is certainly the modus operandi of our department at Oregon Health & Science University (OHSU).

With the large IT investments being made in healthcare, public health, and research organizations, along with the need and desire for baby boomers to manage their increasing use of healthcare, I see a bright future for informatics practitioners. The informatician will rightfully take his or her place on the larger healthcare team, delivering needed expertise on the integration and coordination of information for optimizing people's health.

Update on the ONC Workforce Development Program

It's hard to believe that it has only been a little more than seven months since the Office of the National Coordinator for Health IT (ONC) announced the awardees for its Workforce Development Program. A tremendous amount has been accomplished since the grants were awarded on April 2, 2010. Our department at Oregon Health & Science University (OHSU) has played a substantial role in several aspects of the program.

The ONC Workforce Development Program is devoted to building the professional workforce that will help eligible hospitals and professionals achieve meaningful use of the electronic health records (EHR). The program is based on the need for an estimated 51,000 such professionals who will work in twelve workforce roles. Half of these workforce roles are deemed to be trained in six-month certificate programs in community colleges, while the other half are to be trained in 1-2 years in university-based programs.

The overall program consists of four specific initiatives:
1. Community College Consortia - 84 community colleges, grouped into five regional consortia, have been funded to offer six-month certificate programs in the six community workforce roles.
2. Curriculum Development Centers - Because the community colleges do not have curricula for these programs, five universities have received awards to develop curricular components that are to be developed into courses by the community colleges. One of the five universities (OHSU) was additionally designated the National Training and Dissemination Center (NTDC), tasked with developing the Web site for dissemination of the materials and carrying out training activities for community colleges, including a training event that took place in August, 2010.
3. Competency Examination - An examination to test the competencies gained by graduates of the community college programs for the six workforce roles trained in their programs is being developed.
4. University-based Training (UBT) programs - Additional training funds were awarded to nine universities (including OHSU) for longer-term university-based training.

As noted above, OHSU has been playing a major role in the ONC Workforce Development Program. We are one of the five Curriculum Development Centers and also serve as the NTDC. In addition, we are also one of the nine universities funded under the UBT program.

The work in all of these programs has been substantial over the past seven-plus months. Led by the NTDC, the Curriculum Development Centers have delivered the first version of the 20 curricular components, which are available to the community college programs on the NTDC Web site. The NTDC also put on the training event over August 9-11, 2010 in Portland, OR that brought together over 200 community college faculty, the five Curriculum Development Centers, and some of the ONC leadership.

The Curriculum Development Centers project has spawned another exciting project that will, in the long run, benefit the entire informatics field. This is the development of an educational version of the VA VistA EHR system along with teaching exercises in using and configuring EHRs. The lack of EHR access has always been a problem in informatics education, as vendors have been reluctant to make their systems more readily available. (This is in contrast to companies like IBM and Apple that make development tools available to computer science students for free or at substantial discounts, which makes sense, as it trains a new generation of students in their wares.) This project will rectify this problem and hopefully get vendors to consider ways to be more open with their systems for educational purposes.

OHSU is likewise very busy with its UBT training grant, which will fund 135 Graduate Certificate and 13 Master of Biomedical Informatics over three years. The certificate students are admitted on a rolling basis each quarter (OHSU is on an academic quarter system) and must make the commitment to complete the program in one year. While not requiring full-time study, this pace does require more concentrated commitment than typical part-time students. The program is on-line with no on-campus requirement. We have enrolled 47 certificate students to date and plan to continue admitting 20-30 students per quarter until the funding is completely committed (probably in early 2012). Unfortunately, the number of qualified applicants has vastly exceeded the resources of the grant (about 100 applicants per quarter), although most are offered admission on a self-funded (i.e., paying tuition) basis.

The master's students are admitted in two cohorts, one that started in the fall of 2010 and another that will begin in the fall of 2011. This program is full-time and on-campus.

One novel feature of both programs is the requirement of a practicum (certificate) or internship (master's). We believe that hands-on, real-world experience is essential for learning informatics, even those in on-line programs. We had experience in remote students doing practicums and internships prior to the grant and feel ready to scale up to all of the students funded by the UBT grant. To this end, we have hired a Practicum/Internship Coordinator who will handle the organization and logistics of the program.

We also plan to use the resources of the grant to hire a Career Counselor. This addresses another challenge that informatics programs have faced, which is having the resources to provide career guidance. We have found that generic career advising centers can be helpful to a point, but there is much advice specific to informatics, especially for the students of diverse backgrounds and interests who enroll in our program.

We also hope that in the long run, these enhancements will carry over into our regular program, especially once the ONC UBT grant ends in 2013. I am optimistic that informatics education will continue to be pursued by a variety of students to carry on the work that will not end once the HITECH program finishes. Informatics is and will continue to be a great career option for those wanting to work at the intersection of health disciplines and information technology. As I always say, informatics is more than EHRs, and there will also be continued opportunities in other areas in bioinformatics, other aspects of clinical informatics, public health informatics, and more.

Selected to the Informatics All-Star Team!

This past week I was awarded with a gratifying honor. I was named by Modern Healthcare magazine as one of the Top 25 Clinical Informaticists in the US. I am truly honored to be among this group, which I view as a kind of all-star team for clinical informatics. I am also grateful to OHSU student Paul DeMuro who nominated me for the honor.

Unfortunately, the article describing the awardees in detail is password-protected. However, the list of the 25 awardees and a brief statement about them, including me, is freely accessible.

A big part of this honor has to do with the fact that I am, as the magazine notes, the only full-time educator among the 25. (There are a few other academics, such as Blackford Middleton and Chris Longhurst.) I am delighted that the award committee chose to recognize the value of informatics education as an important part of clinical informatics.

This is actually not my only recent appearance on a "top" list. A blog called HealthTechTopia named me among the Top 10 Most Influential Informatics Professors.