Search Industry Titans and Academic Researchers Converging This Summer in Portland for Information Retrieval Research Conference

Although search is ubiquitous in this age of broadband Internet and mobile wireless devices, and search engine companies are among the most prominent icons of the Internet, there are still many challenges to overcome and new functionality to be developed for search systems. The field of information retrieval (IR) long predates the mainstreaming of search and developments such as the name of the leading search engine becoming a verb (i.e., “Googling”). This field has studied and evaluated the systems and algorithms that established the foundation for modern systems.

The leading research conference in IR, spanning three and a half decades, is the ACM Special Interest Group on Information Retrieval (SIGIR) Conference. The 35th Annual ACM SIGIR Conference will be held this year in Portland, Oregon from August 12-16 at the Portland Downtown Waterfront Marriott. Registration for the conference is now available.

The SIGIR 2012 meeting begins on Sunday, August 12, 2012 with a day of Tutorials, some of which are half-day and two of which are full-day. Also taking place on that day is the Doctoral Consortium, an event that is limited to doctoral students who have been selected to participate. The day finishes up with a Welcome Reception at the conference hotel.

The first day of the regular conference is Monday, August 13. After a breakfast for newcomers to SIGIR, the Opening Ceremony will launch the conference. At this ceremony will be the presentation of the winner of the triennial Gerard Salton Award, who will give a plenary talk. This will be followed by Paper presentations in three simultaneous tracks through the rest of the day. Monday evening will cap off with Posters and Demos, along with a reception.

On Tuesday, August 14, the day will begin with a second Keynote Speaker. This will be followed by Paper presentations in three simultaneous tracks through the rest of the day. Tuesday evening will finish with the Conference Banquet just down the street from the hotel at the Portland World Trade Center.

The final regular day of the conference is Wednesday, August 15. This day will also Paper presentations in three simultaneous tracks through the day. In addition, a separate Industry Track will feature presentations from various researchers and leaders in the commercial sector. This day will also feature the annual SIGIR Business Meeting, with box lunches provided.

The conference will end on Thursday, August 16 with a day of four workshops open to all attendees.

The local host of the meeting is the Oregon Health & Science University Department of Medical Informatics & Clinical Epidemiology. I am honored to serve as the General Conference Chair, as my career in IR applied in the health and biomedical domain spans more than two decades.

Although this event is not one of the massive trade shows one might find about search and related events, this event will bring about 600 researchers from academia and industry, along with students and others, to Portland. Industry sponsors for the event lined up so far include Microsoft Research, Baidu, Google, eBay, IBM Research, Cambridge University Press, Morgan & Claypool Publishers, and Springer. The conference will draw participants from 30-40 countries.

For more information about the conference and to register to attend it, visit the conference Web site at:
http://sigir.org/sigir2012/

OHSU Graduation: As Always, a Time to Celebrate

This week was Oregon Health & Science University (OHSU) graduation, marking a celebration of accomplishment for students from a wide array of disciplines. It was also a milestone for the OHSU biomedical informatics graduate program, marking our 15th year of the program having graduates. I have always enjoyed attending the graduation ceremony, basking in the success of our graduates as well as the program as a whole. I have missed the ceremony only in those 15 years. Below is a picture of some of the graduates and faculty after the ceremony.

We had our annual department banquet the evening before graduation. This is another event I never miss. We honor all graduates who show up for the event as well as the staff who make success possible for them. This year I flew in from Singapore just six hours before the banquet.

I hope our new alumni will also take advantage of and participate our Alumni Steering Committee, which we have stood up in an attempt to remain engaged with them. I hope we can offer our alumni enduring value long after they complete their studies, from continuing education to networking among their peers. I also hope the alumni will serve as ambassadors to inform others about the rewards of careers in the field and the value of studying at OHSU.

As of this graduation, we have now awarded a total of 455 degrees and certificates to 425 people. (The reason for more people than certificates and degrees is that some have received more than one.) The distribution includes:

• Doctor of Philosophy (PhD) - 11
• Master of Science (MS) in Biomedical Informatics - 71
• Master of Science in Biomedical Informatics (MBI) - 107
• Graduate Certificate in Biomedical Informatics - 266

Some accomplishments of our individual programs are also worth noting. One of our graduates from the MS program and also a National Library of Medicine Postdoc Fellow, Dr. Paula Scariati, won the School of Medicine Best Master's Thesis Award for her thesis entitled, Making choices about breast cancer screening: A decision aid for women between the ages of 38 & 48. Our Office of the National Coordinator for Health IT (ONC) University-Based Training (UBT) Grant had its first six MBI graduates along with numerous Graduate Certificate Program graduates. Our health information management (HIM) track continues to thrive, with nine graduates now having successfully passed the Registered Health Information Administrator (RHIA) certification exam, six of whom are graduates of the UBT Graduate Certificate Program.

Who is an OHSU informatics alumnus? There is no single type of person who can be described. These graduates and students come from heterogeneous backgrounds. The enrollment in all of our programs combined is about 30% physicians, 34% other clinicians, and the remainder from a wide array of other backgrounds. About 6% of our students have an MBA, while 4% have an MPH. But we also have a number of other notable fields represented, including law, biosciences, library and information science, and computer science, to name a few.

Of course, our primary goal is not just to achieve numbers. Rather, we aspire (and believe we have succeeded) in providing an education to a wide diversity of people who will be successful in careers applying information and associated technologies to improve peoples' health. To that end, I am personally gratified that our program has touched so many lives and enabled individuals to launch successful careers in biomedical and health informatics.

Disruptive Innovation Coming to Higher Education? The Role of Massive Open Online Courses

The notion of disruptive innovation was popularized by Clayton Christensen [1, 2], and is described as change, usually technological, that causes upheaval of an entire industry sector. We have seen plenty of disruptive innovations in the modern digital era, as the marketplace for products such as books, newspapers, photography, banking, and travel has undergone profound change. Who takes pictures using film or regularly walks into a bank anymore? Who does not spend at least part of their reading time doing so on electronic devices, increasingly those held in the hand, such as smartphones or tablets?

There is a certain irony for those of us who work in academic biomedical and health informatics. On the one hand, we are immersed in the technologies that have caused so much disruptive innovation, i.e., computers, the Internet, and the World Wide Web. On the other hand, those of us in academic informatics apply our work at the intersection of two fields that may be the lone remaining holdouts for disruptive innovation, namely healthcare and education.

We can debate in another post whether disruptive innovation will ever come to healthcare. There are some signs, but I am not holding my breath. Recent developments in higher education, however, potentially portend profound change coming. Being in higher education for a livelihood, I naturally have great interest in the consequences of disruptive innovation within it.

This potential disruptive innovation in higher education comes in the form of what some call massive open online courses (MOOCs). This area has received a great deal of attention lately with the foray of some of the leading US universities into this area, namely Stanford, Harvard, and Massachusetts Institute of Technology (MIT). It has garnered attention in the popular media [3-6].

As most readers of this blog know, I have great enthusiasm for online learning. A good deal of my work in the last decade has focused on the fusion of educational technology with biomedical and health informatics [7-10]. However, the result has mostly been education based on the traditional model of the professor teaching and interacting with a relatively modest number of students.

MOOCs change the calculus of online learning in a much more profound way. Stanford computer science professors Andrew Ng and Daphne Koller have been at the forefront, adapting and delivering their courses to massive audiences [4, 6]. They are part of a new technology venture led by Stanford and including several other big-name US universities called Coursera. Not to be left out, Harvard and Massachusetts Institute of Technology have also launched a similar initiative.

Despite their high profiles, these are not the first such initiatives to disseminate high-quality higher education content via the Web. Two other initiatives, Udacity and the Khan Academy, have been doing this for several years. Resources like the University of Pittsburgh Epidemiology Supercourse have been in existence even longer.

Will these MOOCs lead to disruption in higher education? The cynic in me notes that Ng and Koller are not changing the core Stanford product, where a small number of highly smart students pay a substantial amount of money in the form of Stanford tuition for the privilege of being on the Palo Alto campus and getting a degree from Stanford. I also note that these courses are mostly basic courses, and not the more advanced knowledge that might help someone apply this information. The content is "open" in the sense of being available to anyone, but not in the "wiki" sense of being improved upon in a massive way.

But the optimist in me with the goal of spreading knowledge via technology cannot help but be impressed at the uptake and reach of these courses. I certainly enjoy the global interaction I have through the various educational activities in which I take part in on the Internet. Even Facebook can sometimes be a platform for disseminating knowledge and doing what I enjoy most as an educator, which is getting people to both delve into deeper layers of fact as well as apply them in larger contexts and intellectually principled ways.

As is often the case, the ultimate reality will likely fall somewhere in the middle. Clearly the Web provides an unprecedented vehicle for knowledge dissemination. But education is so much more than a student absorbing knowledge. There is also the in-depth application of that knowledge for real-world purposes. I cannot help but wonder, for example, whether the Coursera natural language processing (NLP) course will enable a student to be able to implement a system that can detail with all the nuances of the narrative text generated by clinicians in the electronic health record. One thing that clinical informatics has taught us is the lack of predictability of technological interventions in healthcare settings.

Of course we have shown to our satisfaction at Oregon Health & Science University (OHSU) that pretty much all types of learning can be delivered online. But we have also learned that an education involves more than learning. Early on in our foray into distance learning, I was struck how we had developed, without deliberately trying to do so, a virtual community. When students join our program, they not only get access to our courses, but also our faculty, their student colleagues, and our connections to the larger informatics world, including our connections to industry. Even the staff in our office provide a conduit for their new journey into careers and other activities in the field.

But I am also, in a sense, part of this MOOC world, due to the Office of the National Coordinator for Health IT (ONC) Curriculum Project that has absorbed a great deal of my professional time, effort, and passion over the last couple years. All of this potential for disruptive innovation of informatics education therefore comes at a time of critical juncture for our field. We have been fortunate to have, for the first time in the history of our field, substantial federal investment, not only in the form of subsidized education for students, but also in the development of the ONC curricular materials. The verdict is still out on what impact the curricular materials will have on informatics education and training in the long run. But with the ARRA funding for them winding down, we are at a critical juncture in finding ways to sustain them (if we believe they are important) once the grant for them ends at the end of 2012.

In conclusion, I view the potential for disruptive innovation in higher education as a challenge and an opportunity. While I am not worried it will make my world dissipate like camera film or bank tellers, I do know the ride will be bumpy. But in the end, I am confident that education will be improved and possibly more cost-effective. I am also confident of the continued role I will play in advising students and others about directions and opportunities for our field. And if things ever do settle down, we can move on to the real challenge for disruptive innovation, which is the healthcare industry!

References

[1] Christensen, C. (1997). The Innovator's Dilemma: When New Technologies Cause Great Firms to Fail. Boston, MA. Harvard Business School Press.
[2] Christensen, C. (2012). Disruptive Innovation, in Soegaard, M. and Dam, R., eds. Encyclopedia of Human-Computer Interaction. Aarhus, Denmark. The Interaction-Design.org Foundation. http://www.interaction-design.org/encyclopedia/disruptive_innovation.html.
[3] Lewin, T. (2012). Instruction for Masses Knocks Down Campus Walls. New York Times. March 4, 2012. http://www.nytimes.com/2012/03/05/education/moocs-large-courses-open-to-all-topple-campus-walls.html.
[4] Markoff, J. (2012). Online Education Venture Lures Cash Infusion and Deals With 5 Top Universities. New York Times. April 18, 2012. http://www.nytimes.com/2012/04/18/technology/coursera-plans-to-announce-university-partners-for-online-classes.html.
[5] Brooks, D. (2012). The Campus Tsunami. New York Times. May 3, 2012. http://www.nytimes.com/2012/05/04/opinion/brooks-the-campus-tsunami.html.
[6] Friedman, T. (2012). Come the Revolution. New York Times. May 15, 2012. http://www.nytimes.com/2012/05/16/opinion/friedman-come-the-revolution.html.
[7] Hersh, W., Junium, K., et al. (2001). Implementation and evaluation of a medical informatics distance education program. Journal of the American Medical Informatics Association, 8: 570-584.
[8] Hersh, W. and Williamson, J. (2007). Educating 10,000 informaticians by 2010: the AMIA 10×10 program. International Journal of Medical Informatics, 76: 377-382.
[9] Hersh, W. (2007). The full spectrum of biomedical informatics education at Oregon Health & Science University. Methods of Information in Medicine, 46: 80-83.
[10] Hersh, W. (2010). The health information technology workforce: estimations of demands and a framework for requirements. Applied Clinical Informatics, 1: 197-212.

ONC Health IT Curriculum: Version 3 and Beyond

Although my last update of the ONC Health IT Curriculum project was relatively recently, there is much news to report, warranting another posting. Some background information for understanding some of the details in this posting in available in a prior posting announcing the availability of Version 2 of the curriculum. It should be remembered that while these curricular materials are freely available to anyone, they are really more designed for educators than students. There is nothing to keep any student from anywhere from downloading them, but they are less designed to be a health IT curriculum "out of the box" and more designed for instructors to develop into materials for specific learners, with additional perspectives, exercises, and even wisdom to be added accordingly.

Probably the most important news is the release of Version 3 of the curriculum. This will be the final version of the curriculum released under the original ONC Health Curriculum Development Centers Program grant. Version 3 is available to anyone to freely download from the National Training & Dissemination Center (NTDC) Web site.

The uncompressed size of the Version 3 materials is 11.2 gigabytes, contained in 18,072 files. As noted in the table below, the 20 components of the curriculum contain 9,974 Powerpoint slides and audio lasting over 136 hours (5 days, 16 hours, and 4 minutes, to be precise!). The NTDC search engine for the text-based files contains 38,181 unique words indexed. A manually constructed topical index is also available.

Of course, not all of the curriculum consists of narrated slides. There are also exercises, including those involving hands-on use of an educational version of VistA for Education (VFE), a fully functional version of the VA VistA electronic health record system, which is also included with the materials. A screen shot of VFE is shown below.

Some additional good news is each of the five universities in the program have been awarded a no-cost extension (NCE) to continue work on the project. In the case of OHSU, this will extend our grant through December 31, 2012. During the NCE time period, we will continue to provide support via the NTDC as well as prepare updates of the components for which we are responsible. Another activity during the NCE period will be to enhance VFE, including exploring the option of providing a fully open-source version that runs on all versions of Windows. (The current version requires a license for Intersystems Cache, which is only freely available to educational institutions.)

A final activity of the five Curriculum Development Centers during the NCE period will be to explore options for sustainability of the curricular materials beyond the end of the NCE period. We are investigating options to obtain funding to maintain, support, and extend the materials while continuing to make this resource freely available.

Spring Renewal in Information Retrieval

Every spring I get a chance to renew my work in information retrieval (IR, also known as search), the area where I first started my research career over two decades ago. My other interests in informatics policy, workforce development, and education, along with my leadership and administrative work, now tend to crowd out the time I devote to IR research and related activity, but I always stay engaged.

The main reason for having my renewal each spring is the teaching of my course in our graduate program, BMI 514/614 - Information Retrieval. At a minimum, this leads me to refresh the updates on the Web site for my book on IR in health and biomedicine. I also often have an opportunity for students to work on projects of mine, especially since the course usually fits well within the annual challenge evaluations of the Text Retrieval Conference (TREC) or ImageCLEF.

This year has been even more of a renewal than most years. Part of the reason is my serving as General Conference Chair of the ACM Special Interest Group in Information Retrieval (SIGIR) 2012 conference, which will be held this August here in Portland. (More in the blog to follow!)

I have also re-engaged in TREC through helping to organize topic development and relevance judgments for the TREC Medical Records Track, which was launched last year to combine aiming to apply IR tools and techniques to retrieval-related problems in electronic health records.

I also recently happened across some fun IR things on the Web. One is ability to create a Wordle from one's scientific publications in the large SciVal collection (see image below). The words that show up largest should surprise no one! Another is the new ability in Google Scholar to set up a profile for one's scientific work, showing most cited works, one's h-index, and other information.

Witness to a Great Public Health Informatics Achievement

Last week, I had the opportunity to participate in a ceremony at the Centers for Disease Control and Prevention (CDC) announcing their Public Health Informatics Fellowship being recognized as a Registered Apprenticeship by the Department of Labor (DOL). This potentially sets the stage for public health informaticians to become a DOL standard occupational code, which means they would appear in DOL labor statistics. This is good news, and hopefully will lead to DOL recognizing other types of informaticians in their statistics.

I went to the meeting to represent AMIA, and had a chance to deliver these remarks:

My name is Dr. William Hersh, and I am Professor and Chair of the Department of Medical Informatics & Clinical Epidemiology at Oregon Health & Science University in Portland, Oregon. I attend this meeting representing the 4000 members of the American Medical Informatics Association (AMIA), the professional society for health-related informatics, and bring words from our President and CEO, Dr. Kevin Fickenscher, who unfortunately could not be here today.

Today, AMIA recognizes the tremendous accomplishment of the CDC’s Scientific Education and Professional Development Program Office in their collaboration with the Department of Labor. The establishment of CDC’s Public Health Informatics Fellowship Program (PHIFP) as a Department of Labor (DOL) Registered Apprenticeship and the laying of the foundation for a standard occupation code for public health informaticians represent a very significant milestone in the decade-long effort by AMIA and its members and leaders to address the crucial issues of informatics workforce development.

The mere fact that the Department of Labor may soon give informaticians a workforce code is very encouraging.  The designation will help the nation realize its ambitions for supporting a fully-interoperable, data-driven learning healthcare system.  Key to this ambition are informaticians of all stripes, not only public health but also clinical, nursing, and even bioinformatics.

In 2001, I and more than 400 AMIA member experts and thought leaders gathered for the organization’s Spring Congress meeting here in Atlanta to develop a national agenda for public health informatics. The resulting 74 recommendations emerged with themes reflected in the CDC/DOL’s decision. Our stakeholders recognized the need to be engaged in coordinated activities related to public health information. They also forecasted the need for informatics training throughout the public health workforce.

A decade later, AMIA experts revisited the national agenda at the 2011 Spring Congress meeting, where we came up with recommendations supporting the need for informatics workforce development and underlining informatics crucial role in the future of public health and healthcare.

Many of us in AMIA are involved in complementary efforts in workforce development in other areas of informatics. I myself have had the opportunity over the last two years to play key roles in the health IT workforce development programs of the Office of the National Coordinator for Health IT, both training professionals in clinical informatics as well as developing the national health IT curriculum focused initially on community college programs but now freely available to the entire world.

Public health informatics is embedded in these efforts, as those in clinical informatics must comprehend how the public health system can benefit from our federal investment in adoption and meaningful use of electronic health records. This is exemplified at my institution, OHSU, where a CDC public health fellowship graduate serves on our faculty and teaches a course in public health informatics to a predominantly clinical informatics student body.

AMIA members will continue to lead the national discussion on informatics workforce development and on what is needed on the front lines of public health. Together with leadership from the federal government, NGOs, public health organizations, associations and specialty societies and business we know informatics professions will grow.  We are encouraged that the CDC/DOL’s ‘public health informaticians’ designation can open the door for other informatics-related designations to follow.

Informatics Professor Elsewhere on the Web

I have had the opportunity to have my blog-related work featured elsewhere on the Web. Some of these sites get more traffic than my own blog.

One site where I have been having edited versions of my blog posts re-posted is HITECH Answers. All of the postings can be found by searching on the tag assigned to them indicating they are from me. HITECH Answers also features a radio show called MULive, where I was the guest on April 3, 2012. (The audio archive of the show can be accessed by registering or going straight to audio link.)

Another site re-posting some of my blog entries of pertinence to internal medicine physicians is the American College of Physicians. Some of my postings are available on ACP Internist, aimed at all internists, while others are available on ACP Hospitalist, aimed at hospitalist physicians.

Postscript: Shortly after this entry was posted, Dr. Kevin Fickenscher, new President and CEO of AMIA, called out a recent posting of mine.

From Implementation to Analytics: The Future Work of Informatics

I am occasionally asked whether the work of informatics will be "done" when everyone is finishing implementing electronic health record (EHR) systems. Sometimes the query is further qualified by, "once everyone gets their HITECH money."

My answer is always an emphatic "No!" There is no question that some informatics implementation activity may slow down when healthcare organizations are no longer fueled by pursuit of HITECH incentive dollars. These activities may be impacted even further by bottom line woes that are likely to impact healthcare no matter what the outcome of healthcare reform, or whatever other distractions come along, such as ICD-10.

I often further qualify my answer by noting that for many of us, the real interesting work of informatics begins when the EHR platform is in place and we can truly start to do interesting things with the data. These are the so-called "secondary uses" or "reuses" of clinical data [1], things like quality measurement and improvement, improved clinical research, or indeed the "learning health system" first envisioned by the Institute of Medicine [2] and put in the context of the HITECH investment by Friedman et al. [3]. Some call this the "optimization" stage of EHR implementation [4].

One buzzword that is used increasingly in healthcare (and was already in use outside of healthcare over the last few years) is analytics. As with all buzzwords, there is a copious volume of material that has been written. I find a couple books by Tom Davenport and associates [5, 6] to provide good overviews. Davenport is Research Director for a company in Portland called the International Institute for Analytics. A recent primer by The Advisory Board Company, a healthcare consulting firm, gives a good overview of analytics in the context of healthcare [7].  Another recent report comes from PwC, which paints a similar picture of the near future, although (to my content!) describes this as clinical informatics (rather than analytics) [8], The phrase business intelligence is sometimes used to describe this work, and I suspect we will see another phrase, big data, appearing more frequently, especially with the recent Obama Administration initiative in this area [9].

The Advisory Board Company primer nicely paints an overview of the use of analytics and business intelligence in healthcare. They distinguish between different uses of the data, each requiring a higher level of analysis and complexity:

• Descriptive - reporting and querying of data to identify problems and solutions
• Predictive - modeling, forecasting, and simulating outcomes based on the data
• Prescriptive - recommend the best course of action based on the data

Of course, those of us who work in clinical informatics know that gleaning value from clinical data is challenging. Indeed, those who have learned from implementation in the trenches may be best qualified to understand the limitations of their data. As I often say, documentation is not usually the highest priority for busy clinicians. Indeed, it is often what stands between a tired clinician at the end of the day and being able to go home for dinner. Clinical data also suffers from the lack of standards in structure and terminology of data, and it is often fragmented across different systems, both within and across different healthcare organizations.

Nonetheless, the growing platform of electronic clinical data, fueled initially by EHR adoption and now augmented by efforts at health information exchange in the proposed rules for Stage 2 of meaningful use, point the way forward [10]. Regardless of one's political views of healthcare reform, it is clear that the system needs to change to become more accountable and efficient. This will be drawn out with the move to new delivery systems, such as accountable care organizations [11]. Thus, analytics and related activities are the future of clinical informatics, realizing the goal of my definition of the field, which is the use of information to improve individual health, healthcare, public health, and biomedical research [12].

References

[1] Safran, C., Bloomrosen, M., et al. (2007). Toward a national framework for the secondary use of health data: an American Medical Informatics Association white paper. Journal of the American Medical Informatics Association, 14: 1-9.

[2] Olsen, L., Aisner, D., et al., eds. (2007). The Learning Healthcare System - Workshop Summary. Washington, DC. National Academies Press.

[3] Friedman, C., Wong, A., et al. (2010). Achieving a nationwide learning health system. Science Translational Medicine, 2(57): 57cm29. http://stm.sciencemag.org/content/2/57/57cm29.full.

[4] Walker, J., Richards, F., et al., eds. (2006). Implementing an Electronic Health Record System New York, NY. Springer.

[5] Davenport, T. and Harris, J. (2007). Competing on Analytics : The New Science of Winning. Cambridge, MA. Harvard Business School Press.

[6] Davenport, T., Harris, J., et al. (2010). Analytics at Work: Smarter Decisions, Better Results. Cambridge, MA. Harvard Business Review Press.

[7] Adams, J. and Klein, J. (2011). Business Intelligence and Analytics in Health Care - A Primer. Washington, DC, The Advisory Board Company. http://www.advisory.com/Research/IT-Strategy-Council/Research-Notes/2011/Business-Intelligence-and-Analytics-in-Health-Care.

[8] Anonymous (2012). Needles in a haystack: Seeking knowledge with clinical informatics, PriceWaterhouseCoopers. http://www.pwc.com/us/en/health-industries/publications/needles-in-a-haystack.jhtml.

[9] Anonymous (2012). Obama Administration Unveils “Big Data” Initiative: Announces $200 Million in New R&D Investments. Washington, DC, White House. http://www.whitehouse.gov/sites/default/files/microsites/ostp/big_data_press_release_final_2.pdf.

[10] Copoulos, M., Raiford, R., et al. (2012). The Next Chapter - First Look at the Proposed Rule on Stage 2 of Meaningful Use. Washington, DC, The Advisory Board Company. http://www.advisory.com/Research/IT-Strategy-Council/Research-Notes/2012/~/media/Advisory-com/Research/ITSC/Research-Notes/2012/The-Next-Chapter-Stage-2.pdf.

[11] Fisher, E., McClellan, M., et al. (2011). Building the path to accountable care. New England Journal of Medicine, 365: 2445-2447.

[12] Hersh, W. (2009). A stimulus to define informatics and health information technology. BMC Medical Informatics & Decision Making, 9: 24. http://www.biomedcentral.com/1472-6947/9/24/.

Informatics Evidence, Redux

About a year ago there was a big dustup in the informatics field concerning a study published by Romano and Stafford in Archives of Internal Medicine that purported to show that electronic health record (EHR) use was not associated with improved quality of care [1]. As honest informaticians, we need to take such research seriously, aiming to improve what we do based on the evidence. This study, however, was problematic, in that it was based on an older data set not designed for answering questions such as the one asked by Romano and Stafford. A better approach would have been to perform a prospective clinical trial that directly assessed an informatics intervention, one of which was indeed published a few months later that did show improvement in care augmented by use of an EHR [2].

Now comes a similar situation a year later with the publication of a study by McCormick et al. in Health Affairs, which uses the same data source to show that physicians who have access to computerized imaging results (not necessarily via an EHR) have a 40-70% higher likelihood of ordering imaging tests [3]. This study set off a similar conversation about whether we are jumping the gun, especially with regards to the substantial federal investment in EHR adoption through the Health Information for Clinical and Economic Health (HITECH) Act. As with the Romano and Stafford study, this new study set off a lot of debate, including an exchange between the National Coordinator for Health IT and a rebuttal by the authors.

It is unfortunate to have to reiterate that we should be guided by the evidence, but given that many of us do have careers staked on the success of the HITECH Act, we must acknowledge potential biases and be as objective as possible in evaluating research results. That said, the study by McCormick truly uses a very weak methodology and certainly does not justify the sweeping conclusions by the authors in their paper or their rebuttal.

Similar to the Romano and Stafford study, this study makes associations with data sources that are not really designed to answer the question of whether EHRs will reduce test ordering. Of course, an even more fundamental question is whether reduced test ordering is something we desire anyways. While imaging tests are clearly over-utilized in healthcare [4,5], this study is incapable of telling us the value of the imaging that was ordered in increased amounts by physicians with access to electronic results. As such, we have no clue as to whether the imaging may or may not be warranted, or how the increased ordering impacted care of the patients for whom it was ordered. It is entirely possible that the increased imaging was beneficial in the management of those patients.

The authors conclude at the end of their abstract that "use of these health information technologies, whatever their other benefits, remains unproven as an effective cost-control strategy with respect to reducing the ordering of unnecessary tests." This is indeed a broad, sweeping conclusion that is hardly warranted from the methods or results of their study.

The rebuttal by Dr. Mostashari is reasonable, noting that the results of this study tell us little of the value of EHRs and the HITECH investment, which is what the authors seem to want to criticize in their results. This certainly comes out in their rebuttal, where they use a number of adjectives to impugn his motives. I do agree with their call for dialogue as well as well-designed clinical trials to assess the benefits of health IT. It may well be that the results of their research are true, and that EHRs will increase costs by making the ordering of expensive tests easier. But we really need our research to answer the larger questions of the value to patient outcomes. We also need to explore how larger changes in our healthcare system, particularly a reimbursement approach that favors quantity over quality, will be empowered by better information systems. With research focusing on those questions, we will be able to ascertain the true value of health IT and how we can improve our use of it.

References

[1] Romano, M. and Stafford, R. (2011). Electronic health records and clinical decision support systems: impact on national ambulatory care quality. Archives of Internal Medicine, 171: 897-903.

[2] Cebul, R., Love, T., et al. (2011). Electronic health records and quality of diabetes care. New England Journal of Medicine, 365: 825-833.

[3] McCormick, D., Bor, D., et al. (2012). Giving office-based physicians electronic access to patients' prior imaging and lab results did not deter ordering of tests. Health Affairs, 31: 488-496.

[4] Baker, L., Atlas, S., et al. (2009). Expanded use of imaging technology and the challenge of measuring value. Health Affairs, 27: 1467-1478.

[5] Hillman, B. and Goldsmith, J. (2010). The uncritical use of high-tech medical imaging. New England Journal of Medicine, 363: 4-6.

Update on the ONC Health IT Curriculum Project

It has been a while since I provided an update of the Office of the National Coordinator for Health Information Technology (ONC) Health IT Curriculum Project. I had the opportunity to give a presentation about the curriculum at this week's HIMSS Conference, so will use the preparation for that to give an update here.

The major news from the project is that the third version of the curriculum will be released in the next month. Version 3 will have the same component names and structure, but the content has been substantially revised and improved. In addition, there will be much more consistency of the slide formats as well as file content and naming. The content itself has been revised based on feedback obtained by a variety of mechanisms, including contracting with the American Medical Informatics Association (AMIA) and expertise they garnered in a process last summer. The materials also have improved accessibility for those with disabilities.

Some have expressed some concern that the project "ends" on April 2, 2012. While it is true that the ONC grant ends on that date, the Web site will continue to be available beyond then. ONC is also considering a no-cost extension of the grants. Stay tuned for more details.

As with Version 2, the Version 3 materials will be made available to the general public. Anyone will be able to go to the Web site of the National Training & Dissemination Center (NTDC) Web site and create a login to enable downloading of the materials.

Other news includes a mention of the curriculum as one of the major accomplishments of ONC for 2011, according to the National Coordinator, Dr. Farzad Mostashsari.

Another useful accomplishment was the addition of a search capability to the NTDC Web site. The search engine allows searching over all text-containing documents. The search engine output allows list the files containing the search terms and allows downloading of the individual file or the unit .zip file that contains the file. The search engine indexed the 1342 Word documents and 460 Powerpoint files and has made them available for word-based searching. (For language trivia buffs, there are 37,485 unique words in these files.)

Additional news about the project includes data about the size of the Version 2 materials as well as download data since its release, including public users.

The entire collection of materials, including the slides, voice-over narration of the slides, and other materials, is 7.84 gigabytes in size. There are a total of 33,172 files. This actually does not include the VA VistA for Education electronic health record system, which has an installer file that is another 770 megabytes in size. VistA requires a license for the Intersystems Cache system, which is freely available to academic institutions but not others. The narration of the slides, available as both Flash-based "video" as well as MP3 audio files, totals 125.6 hours. As noted above, the materials have 1342 Word documents and 460 Powerpoint files. The latter contain a total of 8913 slides.

We also have details about the downloading of Version 2, covering the period from the public rollout to the end of 2011, about one-half year. Before delving into detail about the downloads, it is important to remember the structure and contents of the curriculum. The curriculum consists of 20 components, each aiming to be comparable in size to a three-credit college course. These courses are part of the various workforce roles around which the ONC community college workforce development program is organized, but of course can be used independently either as a whole course or even broken into parts. Each component is broken down into 8-12 units. Each unit contains voice-over-Powerpoint lectures (with transcripts), self-assessment quizzes, and other learning activities (such as discussions and hands-on exercises).

The NTDC web site is structured for downloading by units. The workforce for someone downloading is to create a login (or, in the case of community college faculty users, have a login created, which allows access to additional curricular support) and then navigate through the components to the individual units (packaged in .zip files) for downloading. (We do plan to implement the ability to download entire components in 2012.) Also available for downloading is .zip file containing all the component blueprints (syllabus-like documents) as well as the installer and a help file for VistA for Education.

All told, there were 284,398 downloads of Version 2 units and other files between May-June and the end of 2011. It is important to put this large number in context, which it represents the number of items downloaded. These downloads were carried out by 537 community college faculty and 4680 public users. The public users came from 31 different countries, although the vast majority were from the US. Many of the registering public users did not provide the information the system asked when creating the login, so their background and demographics are not accurately characterized, but browsing of the log shows many educators as well as individuals connected to health care organizations.

The components with the largest number of downloads were Components 3 (22,645), 5 (19,504), and 1 (18,920). The average number of unit downloads per component varied from 2102 for Component 1 to 931 for Component 13. The component blueprints file and VistA for Education installer were downloaded 3255 and 3136 times respectively.

Additional insight can be gained from looking at the minimum and maximum amount of downloads of units within each component. This provides a sense of how many users are downloading one or more units within a component. The minimum number of units downloaded within a component tend to be much closer across the components than the total number of downloads or the maximum. For example, one of the units of Component 5 was only downloaded 925 times, which was not much more than the most minimally downloaded unit of Component 20 (923). This implies that there might be two downloader types: those who take everything for a given component and those who pick and choose.

All told, we are pleased that the ONC Health IT Curriculum has become a substantial global resource. It will be improved with Version 3 that is coming shortly. We are also exploring ways to sustain it beyond the end of the HITECH funding.

eHealth Initiative Report on Hiring in Health Information Exchanges

The eHealth Initiative (eHI), a health information exchange (HIE) advocacy group, recently released a report stating that while HIEs are likely to generate jobs in health information technology (HIT), few of those jobs have gone to those trained by the workforce development programs of the Office of the National Coordinator for Health Information Technology (ONC). As one who is associated with the ONC workforce development programs, I was naturally alerted to the report as well as an article in the trade publication, Health Data Management.

My overall reaction to the eHI report is that while I do not disagree what its findings and conclusions, I do believe those findings and conclusions need to be viewed as part of a larger perspective about HIE and HIT employment. I also believe that the reporting of the methodology used for the report is incomplete, leading to some uncertainty about the meaning of its findings and conclusions. In particular, I wonder whether the report authors or those surveyed fully understand the ONC workforce program or even the HIT workforce itself. There may be more information about this report in one of eHI's proprietary publications, but I cannot find anything on their Web site. (Although I support the work of eHI, I am not a member.)

I do acknowledge up front that I have a vested interest in the ONC workforce development program. I am funded on two grants, one devoted to curriculum development for the six-month community college programs and another for university-based training in the Oregon Health & Science University (OHSU) graduate program in biomedical informatics. I also believe it is fair for anyone to question the value of these programs and whether the investment being made is productive.

My main problem with the report is that its methodology is incompletely described. To begin with, the report itself has no listed author(s) or contact information. Who carried out this report and how can they be contacted?

A related concern is whether those who developed the report's survey or those who answered it are sufficiently knowledgeable about the ONC workforce program itself.  The report does not describe how the question(s) about the ONC programs was/were asked or how knowledgeable the respondents were about the different programs. Many people, for example, are unaware that the program is larger than just the six-month community college certificate programs. Are they knowledgeable, for example, of the university-based training (UBT) programs, which have a workforce role called "Health Informatics Management & Exchange Specialist" that is likely to be most amenable to work in HIEs? This workforce role has been the most subscribed workforce role among the six covered by the UBT programs.

By the same token, the report does not put employment within HIEs in perspective. While I certainly believe that HIEs are a critical element to the larger success of HIT adoption, it is important to remember that the number of people employed in HIEs will be a relatively small part of the overall HIT workforce. For any given state or region, there are many healthcare organizations whose HIT systems will feed into one or a small numbers of HIEs. Although there will be many important jobs for those who implement, lead, and utilize HIEs, their numbers will be modest relative to the large number of HIT professionals in hospitals, physician offices, and other health-related organizations. It is just simple math.

Also important to remember is that many HIEs are still early in development, where the critical skills are more around planning and development than implementation. It is not surprising to see consultants being heavily used, as opposed to professionals just out of their education programs without a great deal of workforce experience. Related to this, the report seems to look only at direct hiring of ONC workforce program graduates by HIEs. We do not know how many graduates of ONC programs work for consultants, vendors, or others as opposed to being hired directly by HIEs, on which the report seems to focus

We also need to remember that HIE jobs vary in the same way that HIT and clinical informatics jobs do. As such, those trained in the ONC programs might not be a fit for the jobs available in HIE. In fact, it is likely that HIE jobs require a great deal of HIT workforce experience, which ONC workforce program graduates by definition do not have.

The Health Data Management article also goes off on a tangent and raises some issues about the ONC HIT curriculum. Some of these are valid criticisms, but it is also important to remember that these curricular materials are designed for HIT teachers, who are encouraged to use them creatively to offer a meaningful learning experience and not just a rote curriculum out of the box. Some of the community colleges have done this better than others in this regard.

While I applaud eHI for brining the workforce issue in the context of HIE to light, I also believe that their report raises more questions than it answers. I do hope that someone will come forth and explain the details of the report's methodology and its findings. I will certainly make a postscript to this entry in my blog if anyone does so. I also encourage dialogue about the value of the ONC workforce programs and how we can improve them not only for content, but also the employability of their graduates.

Is Medicine an Information Science? Perspective from Physician Time Studies

We tend to think of medicine as a health science or a life science, yet in many ways it is an information science, and may be becoming more so with the growth of data generated in the care of patients. If medicine is indeed an information science, then there is a critical role for  biomedical and health informatics, which is the field that uses information to improve some aspect of health, healthcare, and biomedical research.

A couple years ago I reviewed in this blog  two articles that had recently been published about the role of information in medicine. One article, by Stead et al. posited that the quantity and complexity of information in medicine requires a fundamental paradigm shift from the "power of the individual brain" to the "collective power of systems of brains" [1]. The authors noted that the numbers of facts per clinical decision will likely increase exponentially, especially as our knowledge moves beyond the phenotype to include the genotype (e.g., genomic variation, proteomics, etc.). The second article, by Shortliffe, was published about the same time in a special issue of JAMA devoted to medical education [2]. He noted that while medical education (rightly so) goes to great lengths at teaching students how to assess, interact with, and treat patients, it devotes very little effort to obtaining, using, and analyzing another critical component of medical care, namely information.

What evidence is there that medicine is an information science? After all, most modern knowledge workers - i.e, professionals in financial analysis, aviation, and  marketing to name a few - make critical use of information in their work too. A number of studies have looked at how physicians spend their time, and provide clear evidence that information is critically important to their work. Some might think that physicians spend the majority of their time with patients, such as examining them or performing procedures on them. However, these time studies show that physicians spend more time interacting with information, such as reviewing data and documenting patient care, than interacting directly with patients.

These studies assess the tasks of physician work and the time spent doing them. Some of the tasks primarily involve using information. (It  is unfortunate that others in the healthcare environment have not been studies, but as often happens, physicians are the targets whom researchers have chosen to study.) Enough of these studies have been done to lead Tipping and colleagues to perform a systematic review [3]. In addition, four more studies have been done since the completion of the systematic review by Kim et al [4], Tipping et al. [5], Yousefi [6], and Chisholm et al. [7].

The systematic review points out that the studies are heterogeneous and cannot be group to do something like a meta-analysis. Yet the results are surprisingly consistent. The systematic review develops a classification to which most studies relatively adhere. The studies all measure in some manner "direct" patient care, where the physician interacts directly with the patient. They likewise describe "indirect care" of the patient, where the physician reviews patient data, performs documentation, and communicates with various people, such as members of the care team, the patient and/or their family, insurance companies, and others. Finally, most studies have some sort of "other" category that includes travel (either within a healthcare facility or between them), education, and personal time (such as eating). The systematic review and three of the follow-up studies focused physicians who work on hospital wards (i.e., hospitalists), although one of the more recent studies looked at emergency department physicians [7]. The studies have been somewhat though not exclusively weighted toward academic facilities and physicians in training.

Even with the variation in definition of the categories and tasks within them, the results are remarkably consistent. While the range is wide, most of the studies show that physicians spend about 15-17% of their time in direct patient care. Conversely, they spend about 64-67% of their time in indirect patient care, often relatively evenly divided between reviewing results, performing documentation, and engaging in communication. The tasks of reviewing results and carrying out documentation are clearly information-focused in nature, which means that physicians spend about 35-40% of their time engaged with information. One could also probably argue that aspects of direct patient care are information-focused as well, as the physician is gathering information about the patient. The education component of the other category is of course very information-oriented.

Some additional interesting tidbits come of the individual studies. The newer Tipping et al. study took place in a setting of full electronic health record (EHR) implementation and noted 34% of physician time was spent interacting with the EHR [4]. This study and two others by O'Leary et al. [8] and Westbrook et al. [9] in the Tripping et al. systematic review looked at multitasking, finding it was being done during 16-21% of physician work time. O'Leary et al. also found physicians received 3-4 pages per hour [8], while Westbrook et al. noted an average of 2.9 interruptions per hour [9]. Kim et al. found that the amount of direct care was higher at the beginning of shifts while indirect care was higher toward the end of shifts [5]. They also noted that 7% of physician time was spent in travel within the healthcare facility, wondering whether this might be an area where efficiency of work can be improved [5].

In their study of emergency department physicians, Chisholm et al. noted that somewhat more time was spent in direct patient care (31% for academic settings and 38% for community settings) and less in indirect care (55% for academic settings and 50% for community settings) [7]. They also found these emergency physicians were interrupted on the order of 10 times per hour.

These studies collectively show that physicians in hospitals and in emergency departments spend a substantial amount of their time interacting with information. Going forward, the amount and complexity of information is likely to increase. It will come from diverse sources, such as patients entering data into their personal health record (PHR), clinical data coming being provided via health information exchange (HIE), and the growing amount of data from genomics and related areas. This makes the science of biomedical and health informatics even more critical to the medical field.

References

1. Stead, W., Searle, J., et al. (2010). Biomedical informatics: changing what physicians need to know and how they learn. Academic Medicine, 86: 429-434.
2. Shortliffe, E. (2010). Biomedical informatics in the education of physicians. Journal of the American Medical Association, 304: 1227-1228.
3. Tipping, M., Forth, V., et al. (2010). Systematic review of time studies evaluating physicians in the hospital setting. Journal of Hospital Medicine, 5: 353-359.
4. Tipping, M., Forth, V., et al. (2010). Where did the day go?--a time-motion study of hospitalists. Journal of Hospital Medicine, 5: 323-328.
5. Kim, C., Lovejoy, W., et al. (2010). Hospitalist time usage and cyclicality: opportunities to improve efficiency. Journal of Hospital Medicine, 5: 329-334.
6. Yousefi, V. (2011). How Canadian hospitalists spend their time - a work-sampling study within a hospital medicine program in Ontario. Journal of Clinical Outcomes Management, 18: 159-164.
7. Chisholm, C., Weaver, C., et al. (2011). A task analysis of emergency physician activities in academic and community settings. Annals of Emergency Medicine, 18: 117-122.
8. O'Leary, K., Liebovitz, D., et al. (2006). How hospitalists spend their time: insights on efficiency and safety. Journal of Hospital Medicine, 1: 88-93.
9. Westbrook, J., Ampt, A., et al. (2008). All in a day's work: an observational study to quantify how and with whom doctors on hospital wards spend their time. Medical Journal of Australia, 188: 506-509.

One Patient's View of the Optimal Personal Health Record

In teaching current and future informatics professionals, I often speak about the Internet-savvy baby boomers who will interact more with the healthcare system as they get older, which will likely usher in the era of patient-centered informatics more than anything heretofore. I recently had some activities in this role, which gave me some firsthand thoughts about the personal health record (PHR) and interacting with the healthcare system through the PHR and other Web-based means.

There are many views about the role of the PHR and how it should be optimally used. Should it, for example, be primarily connected (sometimes called tethered) to the electronic health record (EHR) of the organization where one receives most or all of their care. While few people desire a truly standalone PHR (i.e., not connected to any data), some advocate it is more important that we move toward an integrated PHR that can interact with data from many sources, from one's own healthcare system to health-related data they capture, such as diet and exercise logs [1, 2].

I recently had the opportunity to interact with my healthcare provider system (OHSU) and its PHR offering (MyChart, tethered to its Epic EHR system). I am fortunate to be in good enough health to not be a major consumer of OHSU healthcare services, but in these interactions, I did come to realize that I want my healthcare system to provide the same kinds of online services that I routinely use for banking, travel, and consumer purchases (e.g., books, electronics, music, etc.). In this regard, OHSU, like many healthcare organizations, falls short.

My experience showed me that what I really want is not so much a PHR (thought it is part of the mix), but rather the ability to manage my data and information with a PHR as well as the ability to carry out all of my interactions with the healthcare system. This includes everything from appointment scheduling and prescription refills to tracking my personal health.

What led to this interaction was what turned out to be a spurious slightly elevated fasting blood sugar. Although I am not overweight, I do have a family history of Type II diabetes, so this is something important to monitor. I also have a number of other cardiac risk factors, including some that are not modifiable (family history), which I try to mitigate with healthy living, namely diet and exercise.

(My cardiac family history is like a roulette table. I have a maternal grandmother and her father who lived to over 100. My maternal grandfather, on the other hand, died of coronary heart disease in his early 50s. Likewise, my maternal grandparents had diabetes and heart disease but lived into their 80s. My father had coronary bypass surgery just before age 50 but is alive and has been symptom-free over 30 years later. Both my maternal grandfather and my father would likely have their coronary heart diseases treated differently in the modern era, with our present array of medications and procedures such as angioplasty. I note that I am also different from them in that they were both smokers. The question is whose genes for coronary disease I have inherited, which is perhaps something our bioinformatics colleagues will be able to answer in the future.)

I also have mild hypertension and a mixed lipid panel, with normal total cholesterol but a sometimes low HDL. In the process of checking a lipid panel, my physician also ordered a metabolic panel, which included a blood glucose. I have always had a fasting glucose at the high end of normal at around 100.

In MyChart, results are released to patients after being reviewed by the provider. This is probably a good idea, although for more routine things, it might not be, since it delays the patient (including knowledgeable ones like me) from getting their results. My initial glucose (along with my lipid profile, which was originally my main concern in getting the blood drawn) was released within hours of the blood being drawn. I was not so lucky for the follow-up tests.

I was impressed to get an email notification within a few hours after the blood was drawn for the first set of tests directing me to MyChart, where my results and a brief message from my physician were waiting. The results showed a fasting blood sugar of 107, which is classified nowadays as "prediabetes." My physician suggested the next step should be to wait and check it again in three months. However, given my family history and other cardiac risk factors, I wanted to know more. In particular, I wanted to know what a two-hour postprandial glucose and a hemoglobin A1C level would show.

My personal physician is also a professional colleague at OHSU and someone I have known since I arrived there 21 years ago. I chose to contact him through the MyChart messaging functionality, although did not get a reply. So I sent him a regular email, to which he responded promptly and ordered the additional tests. I do know that some physicians have trouble keeping up with the stream of email that comes in via MyChart. I do not blame them as much as I blame our healthcare system that only pays for face-to-face medical encounters  and not overall care of the patient, although perhaps that will change with accountable care organizations (ACOs) [3].

I had the second set of tests done on a Friday morning and was hoping for the same quick turnaround as my other tests. This time, that did not happen, and I did not hear back from my physician until late the following Monday. The time lag was certainly not critical for my health, but I did have a desire to learn the results as quickly as possible. I did receive excellent news. Not only was the fasting glucose 97 this time, but my postprandial glucose was 80 and my hemoglobin A1C was 5.1. Not even a hint of diabetes!

Another encounter with the OHSU health system having nothing to do with MyChart but related to electronic interaction with the health system happened about this same time. As noted above, I also have mild hypertension, which is easily controlled with 10 mg of amlodipine daily (and no doubt my healthy diet and dedicated exercise regimen). I get refills for my amlodipine using the OHSU Mail Order Pharmacy. I can request a refill by sending an email to an address on their web site.  There are several problems with this approach. One is that getting my refill in a timely manner is dependent not only on my remembering to send an email a week or so before I run out, but also the timely processing of my request by the pharmacy, which does not always seem to happen. A modern PHR connected to my health system would send a reminder at the appropriate time that let me order the renewal with a click or two.

Another PHR-related activity with my blood pressure comes from the recent home blood pressure monitoring machine I purchased. I am impressed that it stores my results and, when I connect it to a USB port of my computer, uploads the data to my account in Microsoft HealthVault. Of course, it would be more ideal if this data were integrated with my MyChart account, but that does not yet happen. Speaking of HealthVault, I have to say that although I am not always a big fan of Microsoft software or their business practices, they did get it right with HealthVault. It makes sense to have built a PHR platform rather just an application. I could see in the long run how secure cloud-based storage of all our data, even that in the EHR, would be optimal. (Of course, security and availability would need to be rock-solid.)

As mentioned above, I do try to mitigate my cardiac risk factors with diet and exercise. My diet mostly follows the advice of Michael Pollan, "Eat [real] food, mostly plants, not too much" [3]. My exercise consists of running three days a week and cross-training with weights two days a week. I actually pursue this lifestyle less for future benefits and more for the present, as it gives me more energy and makes me feel better here and now. Any later-life benefits will be a plus. I do track my exercise and weight in a spreadsheet but have never felt compelled to take the time to collect any more detail or enter it online.

All of these experiences made it clear to me that what I want most in my online patient experience is not just a PHR, but rather the ability to manage my data integrated with my interactions with all of my healthcare providers. In addition, I want to be able to handle routine transactions in a modern eCommerce-like manner, such as making appointments and ordering prescription refills online. Some may argue that there is not a business case for healthcare organizations to act this way, since our current healthcare system pays clinicians for doing things and not for providing comprehensive, integrated care. I hope, however, that this is not the future, and that healthcare organizations like OHSU will need to serve its customers online because its Internet-savvy baby boomer customers will come to expect it and might seek care elsewhere if they do not get it.

References

1. Detmer, D., Bloomrosen, M., et al. (2008). Integrated personal health records: transformative tools for consumer-centric care. BMC Medical Informatics & Decision Making, 8: 45.
2. Tang, P. and Lee, T. (2009). Your doctor's office or the Internet? Two paths to personal health records. New England Journal of Medicine, 360: 1276-1278.
3. Fisher, E., McClellan, M., et al. (2009). Fostering accountable health care: moving forward in Medicare. Health Affairs, 28: w219-w231.
4. Pollan, M. (2009). In Defense of Food: An Eater's Manifesto. New York, NY. Penguin.

OHSU Biomedical Informatics Education: By the Numbers

This year, 2012, marks the 17th year of the Oregon Health & Science University (OHSU) Biomedical Informatics Graduate Program. What began with a half-dozen Master of Science (MS) students has grown to one of the largest programs in the country, featuring certificates and degrees, three tracks, and a large distance learning component. While informatics education actually started in 1992 with the awarding of our first National Library of Medicine (NLM) Fellowship Training Grant, we did not launch any degree program until 1996.

Now, 17 years later, the program has awarded 401 degrees and certificates to 378 people. Another 19 have completed the NLM Fellowship program without obtaining a degree, bringing the total of alumni to 397 individuals. These alumni are highly successful by any measure, having taken jobs in industry, academia, healthcare organizations, and other settings. Some have gone on to become successful faculty in the field while many more have gone on to take operational informatics roles in companies, healthcare institutions, governments, and other organizations.

The entire number of individuals who have enrolled in any of our graduate programs since its inception  is 1297. The largest enrollment has been in the Graduate Certificate Program, with 974 enrollees. The total enrollment in the other programs has been 27 PhD, 217 MS, and 240 Master of Biomedical Informatics (MBI). The total number of students in all programs adds up to more than the total number of students because some have taken advantage of the "building block" structure of the program that allows students who are enrolled at one level to carry courses forward to a higher level (subject to time limitations).

The most common instance of advancing from one program to a higher level is moving from the Graduate Certificate to one of the master's degree programs. This has been done by 118 students, 41 of whom have completed the master's degree. Three individuals have advanced from the Graduate Certificate to the PhD (two being master's students along the way), while 15 out of the 27 students who have even been in the PhD program were in a master's program at some point.

The breakdown of degree and certificate graduates of the program is 11 PhD, 66 MS, 88 MBI (also including a half-dozen who completed the degree when it was called Master of Medical Informatics) and 236 Graduate Certificate. While it may seem that the program has a somewhat low graduation rate, it must be remembered that people enroll in graduate programs for reasons other than obtaining a degree or certificate. A not insignificant number of our students are already highly accomplished academically and choose a more "a la carte" path to furthering their education than obtaining a degree or certificate. Another reason for a seemingly low rate of graduation is that the program has witnessed substantial growth in recent years, meaning that many students in the program are still current students.

Students in the program have always come from a diversity of backgrounds and many bring substantial prior achievement into the program. More complete data about student backgrounds is available for those who enrolled in the program since 2007, when OHSU started doing a better job of capturing such data.

Of the 819 individuals who have enrolled in the program in 2007 or later, a total of 215 (26.3%) have prior master's degrees, 56 (6.8%) have non-medical doctoral degrees, and 211 (25.8%) have medical degrees. The most common types of master's degrees include MS (82), MBA (40), and MPH (27). The program has 26 (3.2%) individuals with explicit nursing degrees, although we know there are some other nurses who do have "nursing" explicitly in their degree titles.

Another group into which to drill down further is physicians, especially in light of the newly designated subspecialty of clinical informatics and the possibility of altered pathways for training in the future (i.e., clinical fellowships). Our physician enrollment since 2007 includes 184 MD, 15 MBBS, and 12 DO over the course of the program, with 78 MD, 6 MBBS, and 1 DO graduating. These numbers represent 25.8% of the enrollment and 30.7% of the graduates in the program. The MBI program has the highest proportion of physicians, with physicians representing 41.3% of enrollees and 38.3% of the graduates. Physicians in the program tend to be established in their careers and taking on informatics as a new career, with many already reporting professional activity in the field. The average age of physicians in the Graduate Certificate program is 47.8 years and in the MBI program is 44.2 years.

The age distribution of students enrolled since 2007 shows that those in the programs of shorter duration (Graduate Certificate and MBI) tend of be of higher age than the longer programs (MS and PhD). The average age of students in the different programs respective is Graduate Certificate 44.0, MBI 43.6, MS 35.6, and PhD 39.2.

The majority of students in the program since 2007 pursue their studies on a part-time basis. In the Graduate Certificate Program, the average duration of enrollment (including current students) is 1.5 years and the average time to graduate is 2.6 years. In the MBI program, the average duration of enrollment is 3.1 years, while the average time to graduate is 3.9 years. These numbers tend to be relatively comparable for groups with different backgrounds, i.e., those with doctoral degrees, master's degrees, or medical degrees.

The numbers reported so far represent the entire program. As noted at the onset, the program actually has three tracks. While the clinical informatics track dominates the numbers, it is important to note the other two tracks in the program.

The second track in the program is the bioinformatics and computational biology (BCB) track. This track is available on-campus only, and has subject matter that is more computational in nature. Since the launching of this track in 2007, a total of 31 students have matriculated, with 24 in the master's degree programs and seven in the PhD program. (One PhD student had a previous MS from the clinical informatics track.) The program has had 11 graduates, eight with a master's degree and three with a PhD. (One master's graduate completed both the clinical informatics and BCB tracks.) This track is likely to see growth for a variety of reasons, including from the growing role of genomics and related areas in healthcare and biomedical research as well as OHSU's continued investment in quantitative biosciences.

The third track is the health information management (HIM) track, which was launched in 2008. The motivation for this track was to bring about more integration of the HIM and clinical informatics fields, and the curriculum has been structured accordingly, with students combining classes from the clinical informatics track as well as those specific to HIM. The program is accredited by the Commission on the Accreditation of Health Informatics and Information Management (CAHIIM), which allows students to sit for the Registered Health Information Administrator (RHIA) certificate. Since inception, a total of 56 students have enrolled, 17 of whom have graduated. Of the graduates, five have sat for and passed the RHIA certification exam. Just as the American Health Information Management Association (AHIMA) is calling for the HIM entry level to move to the master's degree level, we are expanding the HIM track to the master's degree programs and seeking CAHIIM accreditation at that level.

One final part of our program is the 10x10 ("ten by ten") program. This program was started in partnership with the American Medical Informatics Association (AMIA) in 2005, when then-AMIA President Charles Safran called for one physician and one nurse to be trained in informatics at each US hospital. OHSU operationalized this definition to become 10,000 individuals trained in informatics by 2010 (hence "10x10") and became the first partner with AMIA to offer such a course, which is a standalone version of the introductory course (BMI 510) in the clinical informatics and HIM tracks of the graduate program. At the end of the course, students can optionally take the BMI 510 final exam. If they receive a B grade or better, they can then receive graduate credit for the course upon matriculating in one of the graduate programs without paying additional tuition.

At the end of 2010, a total of 999 individuals completed the OHSU 10x10 offering. Because of continued interest in the course, we have continued to offer it, and by the end of 2011, 1169 people have now completed it. Of those people, 522 (44.7%) have passed the optional final exam, and about 15% gone on to enroll in one of the graduate programs (usually the Graduate Certificate but sometimes the master's program). One individual has "run the table" of OHSU education, starting in 10x10 and advancing to the Graduate Certificate, MBI, and PhD programs.

Secondary Use of Clinical Data, the TREC Medical Records Track, and "Big Data" in Biomedicine

Last week I had the opportunity to present my latest research activity at the OHSU Biomedical Informatics Conference we hold almost every Thursday during the academic year. I chose to present work about the TREC Medical Records Track and its place in the larger context of "secondary use" of clinical data in electronic health record (EHR) systems [1]. The impetus for this work grows with the increasing adoption of EHRs under the HITECH Act, along with the vision of the "learning healthcare system" [2].

I will not recapitulate the talk here, which covers the rationale, data, methods, and early results of the TREC Medical Records Track. (Details can be found on the video and slides from the talk.) I will, however, explore the relationship of what is increasingly called "big data" to biomedicine. One can easily find volumes of information on the Web about big data, but the vision is probably best articulated in the book, The Fourth Paradigm: Data-Intensive Scientific Discovery, published in 2009 by Microsoft Research [3]. This book presents visionary essays on how the growing amount of big data, from EHRs to biomolecular data to patient-entered data will facilitate new discovery of knowledge that conventional experiments will not. As other non-medical essays in the book show, this approach has led to many discoveries in other disciplines that use this form of eScience. We also know that businesses and others make productive use of the vast troves of data they collect from purchases, Web chatter, and other sources of information.

It is important to remember, however, that the existence of large volumes of electronic data does not guarantee that this data will automatically translate into knowledge. In my talk, I reviewed the unfortunately modest amount of literature on this topic. The bottom line, discussed and referenced in more detail below, is that medical records are not only incomplete, but they are also often much less meticulously kept than research data. As I have said in the past, clinical documentation is often what stands between the clinician's daily work and his or her going home for dinner. Another problem with medical records of course is that the data are observational and not experimental, so confounding factors can influence conclusions that might be drawn.

In preparing for this talk, I came across a somewhat obscure but well-written critique of big data [4]. As often happens, I found this paper almost by accident, being pointed to it by one of the email lists to which I subscribe. The primary author of the paper is Danah Boyd, who is another member of Microsoft Research and is also Research Assistant Professor in Media, Culture, and Communication at New York University as well as Visiting Researcher at Harvard Law School. (The paper was delivered as a keynote address at the Oxford Internet Institute's A Decade in Internet Time: Symposium on the Dynamics of the Internet and Society on September 21, 2011.)

Boyd and her co-author list six "provocations" for big data, which sum up to the best critique of big data I have seen. These provocations give us thoughts for concern and are all relevant to biomedicine.  I list them here along with my commentary for applicability in biomedicine or other general comments:

1. Automating Research Changes the Definition of Knowledge - In all research, we tend to meld the question to the data we can obtain. This has certainly been true in biomedical research, where some have criticized research with answering questions either of interest to the research or that have expediency in being able to answer [5, 6, 7]. We need to remember that the data available in electronic systems, big or small, similarly impacts the questions we ask.
2. Claims to Objectivity and Accuracy are Misleading - Just because data are collected in a disinterested way does not mean that bias does not occur. We certainly know from the clinical documentation setting (see above or [8]) that data entered by clinicians is not necessarily accurate, objective, or complete.
3. Bigger Data are Not Always Better Data - This has always been known in medicine from the context of those who do "claims" research based on data collected for billing purposes, which usually consists of diagnosis and procedures codes. One argument for this type of research is the sheer volume of such data, but we also know that this data does not give a complete picture of the patient [9, 10].
4. Not All Data Are Equivalent - We certainly know from the clinical setting that certain types of data (e.g., data collected by motivated researchers) are more likely to be of higher completeness and accuracy than others (e.g., clinical documentation) [11].
5. Just Because it is Accessible Doesn’t Make it Ethical - I agree with the author that the use of Institutional Review Boards is important but also has its limitations in keeping research ethical.
6. Limited Access to Big Data Creates New Digital Divides - I have seen this issue play out in information retrieval research, where the researchers from the big search engine companies have access to proprietary data, which makes peer review as well as reproducibility of the work difficult at best. I know Jimmy Lin personally, and it pains me to read his comment quoted in this paper.

In summary, EHRs provide great potential for improving health and the delivery of healthcare through the learning health system, but we also must remember the caveats of doing so. The consumers of this data need to be cognizant of its limitations but also supportive of the research that explores its more effective use.

References

1. Safran, C., Bloomrosen, M., et al. (2007). Toward a national framework for the secondary use of health data: an American Medical Informatics Association white paper. Journal of the American Medical Informatics Association, 14: 1-9.
2. Friedman, C., Wong, A., et al. (2010). Achieving a nationwide learning health system. Science Translational Medicine, 2(57): 57cm29.
3. Hey, T., Tansley, S., et al., eds. (2009). The Fourth Paradigm: Data-Intensive Scientific Discovery. Redmond, WA. Microsoft Research. http://research.microsoft.com/en-us/collaboration/fourthparadigm/.
4. Boyd, D. and Crawford, K. (2011). Six Provocations for Big Data. Cambridge, MA, Microsoft Research. http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1926431.
5. Harari, E. (2001). Whose evidence?  Lessons from the philosophy of science and the epistemology of medicine. Australia and New Zealand Journal of Psychiatry, 35: 724-730.
6. Cohen, A., Stavri, P., Hersh W. (2004). A categorization and analysis of the criticisms of evidence-based medicine. International Journal of Medical Informatics, 73: 35-43.
7. Tunis, S., Stryer, D., et al. (2003). Practical clinical trials - increasing the value of clinical research for decision making in clinical and health policy. Journal of the American Medical Association, 290: 1624-1632.
8. Benin, A., Vitkauskas, G., et al. (2005). Validity of using an electronic medical record for assessing quality of care in an outpatient setting. Medical Care, 43: 691-698.
8. Jollis, J., Ancukiewicz, M., et al. (1993). Discordance of databases designed for claims payment versus clinical information systems:  implications for outcomes research. Annals of Internal Medicine, 119: 844-850.
9. O'Malley, K., Cook, K., et al. (2005). Measuring diagnoses: ICD code accuracy. Health Services Research, 40: 1620-1639.
10. Berlin, J. and Stang, P. (2011). Clinical Data Sets That Need to Be Mined, 104-114, in Olsen, L., Grossman, C. and McGinnis, J., eds. Learning What Works: Infrastructure Required for Comparative Effectiveness Research. Washington, DC. National Academies Press.

Connecting Informatics Research to Practice: Innovations for AMIA 2012

This year, I will be serving as the Scientific Program Committee (SPC) Chair for the AMIA 2012 Annual Symposium. The annual "AMIA meeting" is the most important biomedical and health informatics scientific meetings of the year, attracting the highest-quality submissions and otherwise providing a snapshot of the field through keynote talks, panels, and other sessions. I am honored to have been selected as SPC Chair for the 2012 meeting and, like many SPC Chairs before me, hope to make some innovations to the meeting that prove to be enduring in value. I was interviewed at the AMIA 2011 conference to give my perspective on the conference and my role in 2012.

The AMIA 2012 innovation I am most excited about is a new category of presentation we are calling the State of the Practice. This session type fits in well with my growing activity at the intersection between the science and practice of informatics. We hope to accept sessions led by experts and leaders from operational settings who will describe key problems and challenges whose solutions have answers in the scientific research of the field. These sessions will provide what all mature professions must have, which is robust and pertinent science that supports operational practice.

Another key AMIA 2012 innovation is a submission category for podium presentations of abstracts. As many AMIA authors and presenters know, the indexing of AMIA papers in the MEDLINE bibliographic database has been a mixed blessing. While it enables authors to have their work made more visible by indexing in the premier biomedical literature database, it also often precludes later, more substantive publication of the work in a scientific journal, due to rules around "prior publication." This new category of submission will allow authors to present their most innovative and cutting-edge work, with the abstract published in the proceedings but not indexed in MEDLINE, so that the author will retain complete flexibility for future publication of the work.

A couple other new changes will be the return of the tutorial program to presenter-initiated submissions (instead of commission by an AMIA committee) and a new pre-symposium program for AMIA Working Groups. The conference Call for Participation provides details on submitting for presentation.

Annual Reflections at Year's End: Reveling in the Successes of 2011 and Looking Ahead

It has become a tradition for me in this blog to post an end-of-year message reflecting on the accomplishments (and, in recent times, thrills) of the past 12 months. This posting follows those from the end of 2009 and 2010.

It has indeed been another incredible year for informatics. Unlike past years, however, we have real accomplishments upon which to report, and not just future dreams. Most of my activity this past year has revolved around projects that are part of the Health Information Technology for Economic and Clinical Health (HITECH) Act that aims to achieve "meaningful use" of electronic health records. This has not, of course, been the main focus of everyone in informatics, as explained further below.

The main activity for me this past year has been carrying the projects that were dreamt about in 2009 and funded in 2010. Many of us still remember spending the winter holiday season of 2009 into 2010 writing proposals for the "Office of No Christmas," aka the Office of the National Coordinator for Health IT (ONC). I also remember the thrill a few months later  upon learning that the two proposals I submitted had been funded, one for curriculum development and the other for training students in our graduate educational program.

There is a joke in academia that the downside of getting grants funded is that you actually have to do the work. However, the work of the ONC projects has truly been a labor of love for me. We have pretty much accomplished everything we said we would, and the results are having a mark on the field. The only sad aspect of these projects is that next year at this time, they will be winding down. We are looking at ways to achieve longer-term sustainability of both.

As noted above, however, not all that is informatics is connected to the HITECH Program. Another major source of activity is in the twin realms of clinical research informatics and translational bioinformatics. Much of this work has been enabled by the Clinical & Translational Science Award (CTSA) Program of the National Institutes of Health (NIH). Informatics has been a prominent feature in the CTSA program, leading to the development of tools and techniques that aid in the use of the data to improve the conduct of biomedical research and ultimately human health. The informatics community has also been well-organized within the CTSA framework. Although my own effort in CTSA has diminished somewhat due to the HITECH work, I am still involved in a number of roles, including working on ways to connect informatics to comparative effectiveness research (CER).

Another important area that is likely to emerge in 2012 and beyond is the informatics of personal health. We can only do so much to improve health care delivery and treatment of disease. Our field needs to pay more attention to maintaining health and preventing disease. To this end, I am pleased to see an exciting new funding opportunity from the US National Science Foundation (NSF) on Smart Health and Well-Being. We still have a lot to learn about health promotion and disease prevention. Those of us who do proactively act on maintaining our health are less prevalent than those who react to disease once it occurs. And of course, some disease just cannot be prevented no matter how healthfully we live.

I am also pleased at year's end that I have been able to sustain this blog. I have preferred to maintain this blog less like many excellent blogs that consist of the blogger's (often well-articulated) stream of consciousness. Instead, I prefer fewer but more focused and developed posts about specific topics, more like a newspaper or magazine column. I plan to continue that approach, and already have many planned postings for the weeks and months ahead. I have been so busy this fall that I have not had time to develop them.

I do wish everyone a healthy and prosperous 2012!