The Internationalization of 10x10

One of the most gratifying aspects of my work in informatics has been its international acceptance. I also enjoy my interactions with international colleagues, both professionally and personally. In addition, I obtain great satisfaction interacting internationally with students, whether they study under myself or others. All told, I enjoy making contributions that are known and valued around the entire world.

As I have written before, I have come to learn that many of the problems faced by informatics are global in nature, i.e., not unique to the United States. All who work in healthcare, public health, and research face challenges in collecting, organizing, and making best use of data and information. All types of information systems present challenges to workflow, usability, and value, among other things.

In recent years, one of my major collaborations has been with colleagues in South America, particularly in Argentina. One of these activities has been for them to translate my well-known 10x10 ("ten by ten") course into Spanish. They have offered the course all across Latin America, with nearly 1000 having completed it. The Spanish 10x10 course uses the same basic approach as my original course, consisting of a series of online units with an in-person session where students meet, learn more together, and/or present projects. While the original version of the Spanish course was a close translation of the English one, the content has now evolved to take on a more Latin American perspective. The core informatics issues are still global, but there are some regional differences, e.g., no delving into the details of HIPAA!

I am pleased to report that the collaboration with Argentina and its translation of the 10x10 course into Spanish has now come full circle, in that the Spanish course has made its way back to the United States via Puerto Rico. As Puerto Rico is fully eligible for funding through the HITECH programs, it has made efforts to help its eligible professionals and hospitals achieve meaningful use of electronic health records. One aspect of this has been the establishment of a regional extension center (REC), the Ponce School of Medicine REC (PSMREC). One of the activities of the PSMREC has been to bring informatics education to clinicians, and it has done this by engaging my Spanish-speaking colleagues and their version of 10x10 to create a new instance of the course, Certificado en Informática Médica para Puerto Rico. The course organizers invited me to participate in their in-person session in San Juan to kick off the course. I spent an enjoyable day with faculty and students, giving talks on the HITECH program as well as the secondary use of clinical data. I was also impressed with the backgrounds of the students taking the course, representing leaders from the REC, academic institutions, healthcare organizations, and others. It is always gratifying as an educator to help those you teach make a difference, which I am certain will happen in this course and what follows in Puerto Rico.

Another active collaboration involving the 10x10 course is in Singapore, with my colleague Dr. KC Lun of Gateway Consulting and the National University of Singapore. This offering of 10x10 uses my English version of the course, with my traveling to Singapore for the end-of-course in-person session. A total of 126 people have completed six offerings of the course over the last several years, with a seventh slated to start this month. I have enjoyed this offering of the course, both through the various people I have met in the Singapore healthcare community as well as the chance to learn about the Singapore healthcare system, which delivers high-quality care at about one-half the per-capita cost of the US.

I also remain active internationally in a number of other ways. I have served for six years as Chair of the Health and Medical Informatics Education Working Group of the International Medical Informatics Association (IMIA). I am a member of the Editorial Board of the International Journal of Medical Informatics. I serve on a number of institutional advisory committees in Europe, Asia, and Africa.

I cannot deny that being able to travel to many of these places has been one of the best perks of my job. Although the real beauty of this travel is that I do not just visit these places as a detached tourist. Rather, I enjoy being welcomed into the work and even home environments of my colleagues. I value seeing firsthand the informatics-related work that they do. I look forward to continuing my work with all of them.

Accolades for the Informatics Professor: Update

As readers of this blog know, I periodically point to other places on the Web where the Informatics Professor has been mentioned or honored. One recent accolade is an article in the magazine, For the Record, describing the recent update to the ONC Health IT Curriculum. I also wrote another article myself for HIMSS Clinical Informatics Insights, a perspective piece about the entire ONC Health IT Workforce Program.

Another accolade is being asked to serve on the (get ready for a mouthful!) Workforce Subgroup of the Certification and Adoption Workgroup of the ONC Health IT Policy Committee. The Health IT Policy Committee sets policy for ONC initiatives, including implementation of meaningful use. The Certification and Adoption Workgroup is tasked with making recommendations "on issues related to the adoption of certified electronic health records that support meaningful use, including issues related to certification, health information extension centers and workforce training." The Workforce Subgroup on which I will be serving is tasked with making "recommendations to the Health IT Policy Committee on ways to provide health IT education to all health care workers."

An additional accolade I have received is being elected as a Fellow of the American College of Medical Informatimusicology (ACMImimi). This honor is bestowed upon those who "see the perfect harmony found in combining healthcare, information technology and music" and manifest it by performing music in a public setting, in my case a recent rendition of some Neil Young songs at the National Library of Medicine Trainees meeting in Madison, Wisconsin on June 26, 2012.

ACMImimi is run by Ross Martin, MD, MHA, an accomplished informatician in many ways, but perhaps most distinctive by his making informatics concepts, especially the HITECH program, accessible via music. I have steered students in recent years to some of his most entertaining and educational pieces:
HITECH: An Interoperetta in Three Acts
The Meaningful Yoose Rap

MOOCs Coming to Informatics?

A few months ago, I wrote a post about the new development of massive open online courses (MOOCs) and what disruptive innovation they might have on higher education. While noting that academic biomedical and health informatics sat at the intersection of the two industries having the least amount of disruptive innovation - healthcare and higher education - I did note that MOOCs could have an impact, if they ever came to our field.

Well now they have, and it turns out that one of my projects is playing a major role in their development. The Health Informatics Forum, an international blogging and social network site for informatics, has started to turn the entire ONC Health IT Curriculum into a MOOC. They recently posted Unit 1 of Component 1 on their site, with announcement of plans to add a new component every four weeks.

How successful will this effort be? There will certainly be value in providing learning materials to the entire world. But there are some caveats. First, as those of us in the ONC Health IT Curriculum project have noted, the materials are designed more for educators than learners. While they provide a rich amount of learning substrate, like all good education they require more, including a teacher, a structured learning process, and ideally fellow learners. In addition, any professional educational experience also requires a connection to the real world through practical opportunities, such as internships. Furthermore, in any rapidly changing field, such as health IT and informatics, the curricular materials must be regularly updated and otherwise improved. It will be interesting to see how sites like The Health Informatics Forum address this latter challenge, particularly as the field evolves. (For example, the Stage 2 meaningful use rules as well as new HIPAA regulations are due out in the next few weeks. As of now, neither of these are covered in the ONC curriculum.)

One final caveat is that the total quantity of these materials represent about 20 college-level courses. This means that any one person will require a great deal of time and effort to work through all of them. By the same token, there are a number of advanced informatics topics that are not covered by the ONC curriculum, such as secondary use of data, natural language processing, and analytics, to name a few. Still, I will be eager to see how this all works out, and hope to lend my expertise to increase its likelihood of success.

It turns out that The Health Informatics Forum is not the only organization that has utilized the totality of the ONC materials as a large learning experience. Two other organizations have done this as well, one of which charges a fee, which is allowable under the Creative Commons license under which the materials have been released:

• http://www.vcn.org/healthcare/online-courses/hit
• http://education.ehrtrainingacademy.com/course/category.php?id=24

The discussion around MOOCs also continues to flourish in the press. The New York Times has run a series of articles, mostly focused on the two efforts led by Stanford (Coursera) and Harvard/MIT (edX) but now expanding to include other universities in their partnerships. This has included articles about the expansion of Coursera as well as the the early experiences of Coursera and edX and one call for caution:

• Expansion of Coursera
• Early experience of Coursera
• Early experience of edX
• One professor's concern about the impersonal aspects of online learning - I actually disagree that online courses need to be impersonal, but I do agree there is no better form of education that the close interaction between the teacher and learner.

It is still too early to tell how these efforts will fare, and what their impact will be on higher education. As one who has been teaching online for 13 years, I can say that learning is very possible, and often desirable, especially when the learner is separated from the learning experience by distance or time. We have many students in our distance learning program at Oregon Health & Science University (OHSU) who desire our education but live far from Oregon and/or work during the time that on-campus classes are offered. In fact, we have a number of "local distance" who live in the area but value the convenience of the online classes.

But our courses at OHSU are anything but MOOCs. They feature direct interaction from our faculty. Furthermore, students can participate in and get credit from structured practicum and internship experiences. This leads one to wonder whether MOOCs might become a means to deliver higher education rather than complete experience in and of themselves. Nonetheless, it will be interesting to see the outcome of this natural experiment in education.

Mapping the AMIA Clinical Informatics Core Content into the OHSU Biomedical Informatics Curriculum

One of the most exciting developments in the clinical informatics field in recent years has been its designation as a new medical subspecialty. Even if one is not a physician, the professional recognition of the work of clinical informatics is important. Hopefully we will see others who work professionally in informatics achieve comparable professional recognition. Indeed, AMIA has established an Advanced Interprofessional Informatics Certification Task Force to explore the best approaches for certification of non-physician informaticians.

Another valuable outcome of the clinical informatics certification process was the development of the core content for the clinical informatics subspecialty (Gardner, R., Overhage, J., et al. (2009). Core content for the subspecialty of clinical informatics. Journal of the American Medical Informatics Association, 16: 153-157. http://jamia.bmj.com/content/16/2/153.full.pdf+html.). This content is by no means limited to physicians and should serve as the basis for the curricular content for all clinical informatics programs.

We recently undertook an analysis of the curricular content in our Biomedical Informatics Graduate Program at Oregon Health & Science University (OHSU) to see how our existing courses mapped into the AMIA core content. We were pleased to discover that just about everything in it is covered by one or more courses in our master's degree program. We are planning to undertake a more detailed analysis in the future to make sure our curricular materials are covered adequately and reflect the most state-of-the-art content. But in the meantime, I am pleased to report that we will be ready to provide education for certification in the field when such programs are ready to be launched. The results of this analysis can be viewed in this PDF file.

Search Industry Track at SIGIR 2012

Although the entirety of the ACM SIGIR 2012 conference, from tutorials to cutting-edge research, will be of interest to those concerned with search systems and algorithms, the Industry Track on the Wednesday (August 15) of the conference will likely be of most interest to those in “tech” industry. In the Industry Track, a series of speakers from all of the major search vendors, most of whom are also sponsors of the conference, will present on their latest works. More information on this track is available, as is information on the conference itself from a local perspective.

The opening speaker of the Industry Track will be Eric Brown of IBM Research, who will present an overview of IBM’s Watson and the DeepQA technology upon which it is built, and explore future applications of this technology. Another speaker will be Andrei Broder of Google, who will discuss the growing field of “computational advertising,” i.e., how algorithms use the context of the user and his or her search terms to display the most appropriate “sponsored” pages. Also speaking will be Daniel Rose of A9.com, a subsidiary of Amazon.com, who will discuss the “democratization” in their new CloudSearch service.

Among the other speakers will be search industry veteran, Sue Dumais of Microsoft, who famously said in 2007 that if search was still using user-entered text boxes in ten years that she should be fired from her job. She will describe the twin problems of putting user context into search as well as putting search into the user’s context. (We are halfway there in 2012 and still using search boxes; maybe someone will ask if she worries for her job!)

Other Industry Track speakers will include John O’Neil of Attivio (Entity Sentiment Extraction Using Text Ranking), Ilya Segalovich of Yandex (Making Web Search More User-Centric: the State of Play and the Way Ahead), and Azarias Reda and colleagues at LinkedIn (Related Searches at LinkedIn).

The track will wrap up with the 2012 Industry Panel consisting of four distinguished panelists who will be asked to represent the likely viewpoint of a particular group or “vertical” while responding to a series of questions notified in advance.  Trystan Upstill of the Search Quality Team at Google will represent “large scale web search”; Jerome Pesenti, Chief Scientist at Vivisimo / IBM, will represent “enterprise search”; Krishna Gade, Engineering Manager at Twitter will represent “real-time and social search.” Stephen Robertson, Emeritus Professor at City University, London and Microsoft Research will represent “academic research” and Diane Kelly, University of North Carolina Chapel Hill and author of an influential monograph on user-involved evaluation, will take the all-important perspective of “users.”  The audience, too, will have their say!

Update on the OHSU ONC University-Based Training Grant

Just as the end of the ONC Health IT Curriculum project is in sight, so is the end of the ONC University-Based Training (UBT) Program. OHSU's UBT grant to train 135 Graduate Certificate and 13 Master of Biomedical Informatics (MBI) students is entering its home stretch.

We have filled all of our UBT slots available for funding, which means that no further students will be funded by this mechanism. We have graduated about half of those who entered the program, i.e., 65 (of 135) Graduate Certificate students and 5 (of 13) MBI students. The remainder will graduate by the end of the grant in June, 2013. As noted in an earlier posting (and additionally below), we have had many success stories from our graduates.

Between the new careers we have enabled our graduates to have, as well as the new positions we have been able to create for faculty and staff in the program, I have no qualms in saying our program is a "job creator."

The UBT program has also been invaluable to the larger OHSU biomedical informatics educational program. It has provided resources allowing the expansion of services by our program, including a practicum and internship program as well as career counseling. In fact, a major goal for our program in the coming year is to find a way to sustain the infrastructure that the ONC grant has allowed us to develop.

Another notable accomplishment for the program was two of our UBT graduates being invited to attend a special two-day event at the Department of Health and Human Services (HHS) and the White House over June 18-19, 2012. The 200 invited attendees included physicians, other healthcare providers, informaticians, vendors, and government officials. The two OHSU attendees were MBI graduate Larry Bannister and Graduate Certificate graduate Dr. Mark Johnson, both of whom reported back as provided below.

Larry Bannister

Mr. Bannister, working formerly in the IT industry, has a position with the Certification Commission for Health Information Technology (CCHIT) in Chicago, IL as a Test Manager. He reported back from the event:

On the first day of the “festivities” here in D.C., we went to the U.S. Dept. of Health and Human Services (HHS). This is housed in the Hubert J. Humphrey Building, which is just a block away from the White House. We started with a  “meet-and-greet” affair for the participants to mingle and for all of us to meet various ONC officials. I met another Oregonian, Dr. DeOna Bridgeman, and we had our picture taken with Dr. Farzad Mostashari. Pretty cool. I also met another OHSU graduate from the biomedical informatics program, Dr. Mark Johnson, and we compared notes about our experiences in the program. After this, we were ushered into a room where you could imagine many a high-powered press conference was given and we heard interesting things about what the ONC is doing to increase the awareness of health IT. Dr. Mostashari gave a very interesting and motivating speech, which was followed by an appearance and speech by HHS Secretary Kathleen Sebelius. The mood throughout the whole event was very upbeat. I know this is an odd thing to say, but I think these folks really, really like what they are doing and seem to be caught up in the excitement of being part of a “movement.” I guess, since I am an HIT person too, I am as well. And, I might add, it is a bipartisan movement, at least from my perspective. Us Republicans see the benefit of HIT and pin many hopes on it as well.

We ended the day by going to various breakout sessions. The session I went to was on “Interoperability & Exchange.” We ranged all over those topics, but the most interesting thing to me was concerning the S&I Framework. This framework, based on the Direct standard, is basically a process for qualifying various entities that want to become participants in an HIE of some kind. Although they have gone down the road a ways, especially in the east coast states, they are also working on another model, the Nationwide Health Information Network, or NwHIN, which is based on the Exchange standard. The ONC recently issued a governance RFI in which they solicited feedback from the HIT community at large concerning NwHIN, and so they solicited comment from us too. However, it was mostly physicians there, with a few policy wonks and just a few technical folks (like me), so although they solicited feedback on the RFI, what the crowd wanted to talk about was:  1) trust issues as the basis of success for HIE’s, 2) payment models that held back most MDs and hospitals from getting on board, and 3) for those MDs and organizations that were onboard, how the current Direct project was so lame and cumbersome (their experience being with portals a.k.a. HISPs). Anyway, they urged everyone to read the “66 questions”  from the RFI and chip in their two cents. I think that is a very good thing for any HIT person to do, as well, since it will probably affect us in the future. After the session broke up we continued to talk amongst ourselves for at least a half hour – everyone was so interested in what others were doing.

On the second day we went to the White House. On that day, ONC announced that over 110,000 eligible physicians and over 2400 eligible hospitals had received meaningful use incentive payments. It was an interesting experience, just getting into the White House. The meeting was held in one of the three (or is it four?) buildings that make up the White House, i.e., the Eisenhower Executive Office Building. Security was tight getting in, just as you would expect, but we had to wait outside in stifling heat and humidity for a while since there were so many of us to process through. My suit was pretty wrinkled and sweaty before I finally sat down in the air-conditioned room where we had our meeting. Just as the day before, we heard some opening comments by Dr. Mostashari and other ONC/HHS officials, but then we had a town-hall session where people from the audience were given time at the microphone and had their say. Dr. Mostashari moderated this and kept us on task and on time.

The topics ranged all over the place so I can’t focus on any one in particular, but they mainly focused on HIE experiences, usability, and war stories. Here are just a few that I took notes on:

• “All RHIOs are not the same” was the opening comment. One MD from Colorado complained about how his HIE experience was dismal and expensive and wondered what could be done better
• A number of physicians related success stories about how they made HIEs work well. Dr. Larry Garber from Massachusetts related how their EHR interfaced to 5 different hospitals, multiple health plans and specialty practices and did such magical things as automatically incorporate both labs and notes from a patient’s ER visit into their medical record
• There seemed to be just as many stories about how peer-to-peer interfaces came up faster and worked better as there were stories concerning coordinated HIE efforts using Exchange.
• A number of calls for standards for things as diverse as: vendor-neutral medical record exchange; history & physical documentation; a single HL7 standard that meant the same thing to everyone;  a standard for alerts and finally, usability standards.
• A number of different funding models for HIE were discussed:  government funding models, payer-funded models and shared payer-physician-vendor split funding models, to name a few.
• Most interesting to me was the story of a group practice that had created a large general –purpose, table-oriented templating system that was easily customizable to produce notes/documentation at any level of detail for each provider’s requirements for their medical documentation.
• Usability popped up again when someone called on vendors to consider usability issues at the very earliest stages of system design and then there was a short diversion into the “geek gap”.  Someone said that EHRs shouldn’t be designed by “propeller heads” (which even though I’m a software engineer, I heartily agree with) but rather, it has to be a well-coordinated, collaborative process.

Anyway, I could not take notes on all that was being said – I was just fascinated and caught up in the moment so often.  It was great and I left feeling really jazzed about the future of this field. There are some exciting times ahead and I’m glad to be part of this thing, myself.

Mark Johnson

Dr. Johnson was formerly a Staff Physician in Critical Care Medicine at the University of Wisconsin Hospital and Clinics. Since obtaining his Graduate Certificate at OHSU, he has joined the Carle Clinic in Urbana, IL as a critical care physician and informatician. Dr. Johnson reported:

It was both an honor and a humbling experience attending events at the White House, ONC, and HHS.  With recent completion of a critical care fellowship at University of Wisconsin-Madison, and a Graduate Certificate in clinical informatics at OHSU (funded by the ONC), I have been uniquely positioned to be a clinician leader in the realm of health IT and the ICU. This led to multiple job offers and acceptance of my dream job (combining health IT and critical care) near family and my alma mater in Urbana, IL. I was thrilled to share my story with others around the nation.

During the flight to DC, I wrote down some of my hopes and worries about the trip. Hoping to: connect with others, share thoughts on redesigning the healthcare system, be a part of the national health IT agenda and synchronize my own career with the timetable and aligned goals. Worried that: I may not fit in, be behind the times, and be too closely tied to a single vendor.

On June 18th, I arrived at the HHS building and mingled with other “Health IT Vanguards,” as the ONC dubbed us. There I met Larry Bannister, another OHSU trainee, and heard about how he was able to retool, combining his IT experience and biomedical informatics training to secure a position at CCHIT, an important entity in the health IT world. I also met a young surgical trainee from Duke, a primary care physician from Oregon, and others. The room had a lot of energy from the “Vanguards” as well as the ONC leadership. I was excited to meet Dr. Farzad Mostashari, National Coordinator for Health IT, and share my story. I thanked him (and ONC) for the training grant.

Next, we were ushered to an auditorium, with speeches from Dr. Mostashari and the Secretary of HHS, Kathleen Sebelius. I was impressed that they seemed tapped into what was going on around the nation. Dr. Mostashari spoke about us as vanguards and our ability to “walk through walls”, to breakthrough barriers to get things done. And he spoke about aiming toward a “healthcare system that does right,” noting, “we don’t always do it right but we always try to do it right.” Secretary Sebelius spoke about how government creates conditions, and creates paths, and that we should share our stories so that others may follow our paths, and to “keep pushing forward.”

These speeches were followed by media training, on how to effectively share our stories with the media. Breakout discussions capped of the event at HHS, and I attended the “Leveraging Health IT for Quality Improvement” discussion. Here I highlighted that there was much knowledge in the room, and across health systems (in regards to how best implement health IT), but there was no system to share this knowledge. Also that ONC can serve as a lever/feedback mechanism to the vendors.

The next morning was the main event at the White House complex. Here was a broader speech by Dr. Mostashari, panelists from around the nation, and a lively Q&A session. Key topics discussed included health information exchange, usability, vendor relations, and “moving beyond the data.” Key quotes that I wrote down included from a vendor “we want ONE standard,” and “ONC needs to hold their [vendor’s] feet to the fire.” From a health IT clinician “culture eats strategy for lunch.” From Dr. Mostashari re: HIE “lots of different models work,” we need to stop “hand fitting pipes,” and set “common rules of the road.”

In summary, the event was enlightening in how government works, what the nation’s agenda is moving forward, and a first-hand sample of what is going on around the country. Most of my hopes leading up to the event were realized and the worries were allayed. I was thrilled to be a part of the event, and feel called to continue working toward a health system that does right. This adventure continues locally in my new position in Urbana, IL and will unfold as I build on my ONC funded training at OHSU.

Below are pictured two of the ONC Health IT Vanguards, Mr. Bannister (left) and Dr. Johnson (right).

The entire ONC Workforce Development Program has been beneficial to our students, our program, and our country. I recently had the opportunity to write an overview of the ONC Workforce Development Program two years into the program for the HIMSS Clinical Informatics Insights publication.

Postscript: The White House visit by the OHSU students was also covered in the online School of Medicine News.

Where Do OHSU Biomedical Informatics Students Live?

We have always known that OHSU students in the distance learning biomedical informatics program live all over the country, and the world, but thanks to Dr. David Dorr and Kelly Jones-Weir, we now have a graphical depiction of their distribution across the United States. Each dot (star) in the figure below represents where one student lives.

The concentration of dots in the Portland area represents an interesting phenomenon. We have a number of what we call "local distance" students in the program, who are people that live in the Portland area yet are enrolled in our distance program. For the most part, these individuals prefer the flexibility that the on-line courses offer. Some may have jobs or other obligations during the day when on-campus classes are offered. One nice aspect is that they take part in other program activities, such as our local research conferences and other events.

In addition to students from the US, we also have students from a number of different countries. Some of these students come to Oregon to study but others are distance learning students (who even come on-campus for our hybrid short courses, as all master's students must do). The countries with one or more students in the program include:

• Argentina
• Australia
• Canada
• Egypt
• India
• Israel
• Kuwait
• New Caledonia
• Saudi Arabia
• Singapore
• Thailand
• United Kingdom
• Zimbabwe

While our program is somewhat US-centric, since those who practice informatics in the US must understand the nuances (some might say craziness) of the US healthcare system, I have also noted in the past that many informatics problems and solutions are global in their nature.

Search Industry Titans and Academic Researchers Converging This Summer in Portland for Information Retrieval Research Conference

Although search is ubiquitous in this age of broadband Internet and mobile wireless devices, and search engine companies are among the most prominent icons of the Internet, there are still many challenges to overcome and new functionality to be developed for search systems. The field of information retrieval (IR) long predates the mainstreaming of search and developments such as the name of the leading search engine becoming a verb (i.e., “Googling”). This field has studied and evaluated the systems and algorithms that established the foundation for modern systems.

The leading research conference in IR, spanning three and a half decades, is the ACM Special Interest Group on Information Retrieval (SIGIR) Conference. The 35th Annual ACM SIGIR Conference will be held this year in Portland, Oregon from August 12-16 at the Portland Downtown Waterfront Marriott. Registration for the conference is now available.

The SIGIR 2012 meeting begins on Sunday, August 12, 2012 with a day of Tutorials, some of which are half-day and two of which are full-day. Also taking place on that day is the Doctoral Consortium, an event that is limited to doctoral students who have been selected to participate. The day finishes up with a Welcome Reception at the conference hotel.

The first day of the regular conference is Monday, August 13. After a breakfast for newcomers to SIGIR, the Opening Ceremony will launch the conference. At this ceremony will be the presentation of the winner of the triennial Gerard Salton Award, who will give a plenary talk. This will be followed by Paper presentations in three simultaneous tracks through the rest of the day. Monday evening will cap off with Posters and Demos, along with a reception.

On Tuesday, August 14, the day will begin with a second Keynote Speaker. This will be followed by Paper presentations in three simultaneous tracks through the rest of the day. Tuesday evening will finish with the Conference Banquet just down the street from the hotel at the Portland World Trade Center.

The final regular day of the conference is Wednesday, August 15. This day will also Paper presentations in three simultaneous tracks through the day. In addition, a separate Industry Track will feature presentations from various researchers and leaders in the commercial sector. This day will also feature the annual SIGIR Business Meeting, with box lunches provided.

The conference will end on Thursday, August 16 with a day of four workshops open to all attendees.

The local host of the meeting is the Oregon Health & Science University Department of Medical Informatics & Clinical Epidemiology. I am honored to serve as the General Conference Chair, as my career in IR applied in the health and biomedical domain spans more than two decades.

Although this event is not one of the massive trade shows one might find about search and related events, this event will bring about 600 researchers from academia and industry, along with students and others, to Portland. Industry sponsors for the event lined up so far include Microsoft Research, Baidu, Google, eBay, IBM Research, Cambridge University Press, Morgan & Claypool Publishers, and Springer. The conference will draw participants from 30-40 countries.

For more information about the conference and to register to attend it, visit the conference Web site at:
http://sigir.org/sigir2012/

OHSU Graduation: As Always, a Time to Celebrate

This week was Oregon Health & Science University (OHSU) graduation, marking a celebration of accomplishment for students from a wide array of disciplines. It was also a milestone for the OHSU biomedical informatics graduate program, marking our 15th year of the program having graduates. I have always enjoyed attending the graduation ceremony, basking in the success of our graduates as well as the program as a whole. I have missed the ceremony only in those 15 years. Below is a picture of some of the graduates and faculty after the ceremony.

We had our annual department banquet the evening before graduation. This is another event I never miss. We honor all graduates who show up for the event as well as the staff who make success possible for them. This year I flew in from Singapore just six hours before the banquet.

I hope our new alumni will also take advantage of and participate our Alumni Steering Committee, which we have stood up in an attempt to remain engaged with them. I hope we can offer our alumni enduring value long after they complete their studies, from continuing education to networking among their peers. I also hope the alumni will serve as ambassadors to inform others about the rewards of careers in the field and the value of studying at OHSU.

As of this graduation, we have now awarded a total of 455 degrees and certificates to 425 people. (The reason for more people than certificates and degrees is that some have received more than one.) The distribution includes:

• Doctor of Philosophy (PhD) - 11
• Master of Science (MS) in Biomedical Informatics - 71
• Master of Science in Biomedical Informatics (MBI) - 107
• Graduate Certificate in Biomedical Informatics - 266

Some accomplishments of our individual programs are also worth noting. One of our graduates from the MS program and also a National Library of Medicine Postdoc Fellow, Dr. Paula Scariati, won the School of Medicine Best Master's Thesis Award for her thesis entitled, Making choices about breast cancer screening: A decision aid for women between the ages of 38 & 48. Our Office of the National Coordinator for Health IT (ONC) University-Based Training (UBT) Grant had its first six MBI graduates along with numerous Graduate Certificate Program graduates. Our health information management (HIM) track continues to thrive, with nine graduates now having successfully passed the Registered Health Information Administrator (RHIA) certification exam, six of whom are graduates of the UBT Graduate Certificate Program.

Who is an OHSU informatics alumnus? There is no single type of person who can be described. These graduates and students come from heterogeneous backgrounds. The enrollment in all of our programs combined is about 30% physicians, 34% other clinicians, and the remainder from a wide array of other backgrounds. About 6% of our students have an MBA, while 4% have an MPH. But we also have a number of other notable fields represented, including law, biosciences, library and information science, and computer science, to name a few.

Of course, our primary goal is not just to achieve numbers. Rather, we aspire (and believe we have succeeded) in providing an education to a wide diversity of people who will be successful in careers applying information and associated technologies to improve peoples' health. To that end, I am personally gratified that our program has touched so many lives and enabled individuals to launch successful careers in biomedical and health informatics.

Disruptive Innovation Coming to Higher Education? The Role of Massive Open Online Courses

The notion of disruptive innovation was popularized by Clayton Christensen [1, 2], and is described as change, usually technological, that causes upheaval of an entire industry sector. We have seen plenty of disruptive innovations in the modern digital era, as the marketplace for products such as books, newspapers, photography, banking, and travel has undergone profound change. Who takes pictures using film or regularly walks into a bank anymore? Who does not spend at least part of their reading time doing so on electronic devices, increasingly those held in the hand, such as smartphones or tablets?

There is a certain irony for those of us who work in academic biomedical and health informatics. On the one hand, we are immersed in the technologies that have caused so much disruptive innovation, i.e., computers, the Internet, and the World Wide Web. On the other hand, those of us in academic informatics apply our work at the intersection of two fields that may be the lone remaining holdouts for disruptive innovation, namely healthcare and education.

We can debate in another post whether disruptive innovation will ever come to healthcare. There are some signs, but I am not holding my breath. Recent developments in higher education, however, potentially portend profound change coming. Being in higher education for a livelihood, I naturally have great interest in the consequences of disruptive innovation within it.

This potential disruptive innovation in higher education comes in the form of what some call massive open online courses (MOOCs). This area has received a great deal of attention lately with the foray of some of the leading US universities into this area, namely Stanford, Harvard, and Massachusetts Institute of Technology (MIT). It has garnered attention in the popular media [3-6].

As most readers of this blog know, I have great enthusiasm for online learning. A good deal of my work in the last decade has focused on the fusion of educational technology with biomedical and health informatics [7-10]. However, the result has mostly been education based on the traditional model of the professor teaching and interacting with a relatively modest number of students.

MOOCs change the calculus of online learning in a much more profound way. Stanford computer science professors Andrew Ng and Daphne Koller have been at the forefront, adapting and delivering their courses to massive audiences [4, 6]. They are part of a new technology venture led by Stanford and including several other big-name US universities called Coursera. Not to be left out, Harvard and Massachusetts Institute of Technology have also launched a similar initiative.

Despite their high profiles, these are not the first such initiatives to disseminate high-quality higher education content via the Web. Two other initiatives, Udacity and the Khan Academy, have been doing this for several years. Resources like the University of Pittsburgh Epidemiology Supercourse have been in existence even longer.

Will these MOOCs lead to disruption in higher education? The cynic in me notes that Ng and Koller are not changing the core Stanford product, where a small number of highly smart students pay a substantial amount of money in the form of Stanford tuition for the privilege of being on the Palo Alto campus and getting a degree from Stanford. I also note that these courses are mostly basic courses, and not the more advanced knowledge that might help someone apply this information. The content is "open" in the sense of being available to anyone, but not in the "wiki" sense of being improved upon in a massive way.

But the optimist in me with the goal of spreading knowledge via technology cannot help but be impressed at the uptake and reach of these courses. I certainly enjoy the global interaction I have through the various educational activities in which I take part in on the Internet. Even Facebook can sometimes be a platform for disseminating knowledge and doing what I enjoy most as an educator, which is getting people to both delve into deeper layers of fact as well as apply them in larger contexts and intellectually principled ways.

As is often the case, the ultimate reality will likely fall somewhere in the middle. Clearly the Web provides an unprecedented vehicle for knowledge dissemination. But education is so much more than a student absorbing knowledge. There is also the in-depth application of that knowledge for real-world purposes. I cannot help but wonder, for example, whether the Coursera natural language processing (NLP) course will enable a student to be able to implement a system that can detail with all the nuances of the narrative text generated by clinicians in the electronic health record. One thing that clinical informatics has taught us is the lack of predictability of technological interventions in healthcare settings.

Of course we have shown to our satisfaction at Oregon Health & Science University (OHSU) that pretty much all types of learning can be delivered online. But we have also learned that an education involves more than learning. Early on in our foray into distance learning, I was struck how we had developed, without deliberately trying to do so, a virtual community. When students join our program, they not only get access to our courses, but also our faculty, their student colleagues, and our connections to the larger informatics world, including our connections to industry. Even the staff in our office provide a conduit for their new journey into careers and other activities in the field.

But I am also, in a sense, part of this MOOC world, due to the Office of the National Coordinator for Health IT (ONC) Curriculum Project that has absorbed a great deal of my professional time, effort, and passion over the last couple years. All of this potential for disruptive innovation of informatics education therefore comes at a time of critical juncture for our field. We have been fortunate to have, for the first time in the history of our field, substantial federal investment, not only in the form of subsidized education for students, but also in the development of the ONC curricular materials. The verdict is still out on what impact the curricular materials will have on informatics education and training in the long run. But with the ARRA funding for them winding down, we are at a critical juncture in finding ways to sustain them (if we believe they are important) once the grant for them ends at the end of 2012.

In conclusion, I view the potential for disruptive innovation in higher education as a challenge and an opportunity. While I am not worried it will make my world dissipate like camera film or bank tellers, I do know the ride will be bumpy. But in the end, I am confident that education will be improved and possibly more cost-effective. I am also confident of the continued role I will play in advising students and others about directions and opportunities for our field. And if things ever do settle down, we can move on to the real challenge for disruptive innovation, which is the healthcare industry!

References

[1] Christensen, C. (1997). The Innovator's Dilemma: When New Technologies Cause Great Firms to Fail. Boston, MA. Harvard Business School Press.
[2] Christensen, C. (2012). Disruptive Innovation, in Soegaard, M. and Dam, R., eds. Encyclopedia of Human-Computer Interaction. Aarhus, Denmark. The Interaction-Design.org Foundation. http://www.interaction-design.org/encyclopedia/disruptive_innovation.html.
[3] Lewin, T. (2012). Instruction for Masses Knocks Down Campus Walls. New York Times. March 4, 2012. http://www.nytimes.com/2012/03/05/education/moocs-large-courses-open-to-all-topple-campus-walls.html.
[4] Markoff, J. (2012). Online Education Venture Lures Cash Infusion and Deals With 5 Top Universities. New York Times. April 18, 2012. http://www.nytimes.com/2012/04/18/technology/coursera-plans-to-announce-university-partners-for-online-classes.html.
[5] Brooks, D. (2012). The Campus Tsunami. New York Times. May 3, 2012. http://www.nytimes.com/2012/05/04/opinion/brooks-the-campus-tsunami.html.
[6] Friedman, T. (2012). Come the Revolution. New York Times. May 15, 2012. http://www.nytimes.com/2012/05/16/opinion/friedman-come-the-revolution.html.
[7] Hersh, W., Junium, K., et al. (2001). Implementation and evaluation of a medical informatics distance education program. Journal of the American Medical Informatics Association, 8: 570-584.
[8] Hersh, W. and Williamson, J. (2007). Educating 10,000 informaticians by 2010: the AMIA 10×10 program. International Journal of Medical Informatics, 76: 377-382.
[9] Hersh, W. (2007). The full spectrum of biomedical informatics education at Oregon Health & Science University. Methods of Information in Medicine, 46: 80-83.
[10] Hersh, W. (2010). The health information technology workforce: estimations of demands and a framework for requirements. Applied Clinical Informatics, 1: 197-212.

ONC Health IT Curriculum: Version 3 and Beyond

Although my last update of the ONC Health IT Curriculum project was relatively recently, there is much news to report, warranting another posting. Some background information for understanding some of the details in this posting in available in a prior posting announcing the availability of Version 2 of the curriculum. It should be remembered that while these curricular materials are freely available to anyone, they are really more designed for educators than students. There is nothing to keep any student from anywhere from downloading them, but they are less designed to be a health IT curriculum "out of the box" and more designed for instructors to develop into materials for specific learners, with additional perspectives, exercises, and even wisdom to be added accordingly.

Probably the most important news is the release of Version 3 of the curriculum. This will be the final version of the curriculum released under the original ONC Health Curriculum Development Centers Program grant. Version 3 is available to anyone to freely download from the National Training & Dissemination Center (NTDC) Web site.

The uncompressed size of the Version 3 materials is 11.2 gigabytes, contained in 18,072 files. As noted in the table below, the 20 components of the curriculum contain 9,974 Powerpoint slides and audio lasting over 136 hours (5 days, 16 hours, and 4 minutes, to be precise!). The NTDC search engine for the text-based files contains 38,181 unique words indexed. A manually constructed topical index is also available.

Of course, not all of the curriculum consists of narrated slides. There are also exercises, including those involving hands-on use of an educational version of VistA for Education (VFE), a fully functional version of the VA VistA electronic health record system, which is also included with the materials. A screen shot of VFE is shown below.

Some additional good news is each of the five universities in the program have been awarded a no-cost extension (NCE) to continue work on the project. In the case of OHSU, this will extend our grant through December 31, 2012. During the NCE time period, we will continue to provide support via the NTDC as well as prepare updates of the components for which we are responsible. Another activity during the NCE period will be to enhance VFE, including exploring the option of providing a fully open-source version that runs on all versions of Windows. (The current version requires a license for Intersystems Cache, which is only freely available to educational institutions.)

A final activity of the five Curriculum Development Centers during the NCE period will be to explore options for sustainability of the curricular materials beyond the end of the NCE period. We are investigating options to obtain funding to maintain, support, and extend the materials while continuing to make this resource freely available.

Spring Renewal in Information Retrieval

Every spring I get a chance to renew my work in information retrieval (IR, also known as search), the area where I first started my research career over two decades ago. My other interests in informatics policy, workforce development, and education, along with my leadership and administrative work, now tend to crowd out the time I devote to IR research and related activity, but I always stay engaged.

The main reason for having my renewal each spring is the teaching of my course in our graduate program, BMI 514/614 - Information Retrieval. At a minimum, this leads me to refresh the updates on the Web site for my book on IR in health and biomedicine. I also often have an opportunity for students to work on projects of mine, especially since the course usually fits well within the annual challenge evaluations of the Text Retrieval Conference (TREC) or ImageCLEF.

This year has been even more of a renewal than most years. Part of the reason is my serving as General Conference Chair of the ACM Special Interest Group in Information Retrieval (SIGIR) 2012 conference, which will be held this August here in Portland. (More in the blog to follow!)

I have also re-engaged in TREC through helping to organize topic development and relevance judgments for the TREC Medical Records Track, which was launched last year to combine aiming to apply IR tools and techniques to retrieval-related problems in electronic health records.

I also recently happened across some fun IR things on the Web. One is ability to create a Wordle from one's scientific publications in the large SciVal collection (see image below). The words that show up largest should surprise no one! Another is the new ability in Google Scholar to set up a profile for one's scientific work, showing most cited works, one's h-index, and other information.

Witness to a Great Public Health Informatics Achievement

Last week, I had the opportunity to participate in a ceremony at the Centers for Disease Control and Prevention (CDC) announcing their Public Health Informatics Fellowship being recognized as a Registered Apprenticeship by the Department of Labor (DOL). This potentially sets the stage for public health informaticians to become a DOL standard occupational code, which means they would appear in DOL labor statistics. This is good news, and hopefully will lead to DOL recognizing other types of informaticians in their statistics.

I went to the meeting to represent AMIA, and had a chance to deliver these remarks:

My name is Dr. William Hersh, and I am Professor and Chair of the Department of Medical Informatics & Clinical Epidemiology at Oregon Health & Science University in Portland, Oregon. I attend this meeting representing the 4000 members of the American Medical Informatics Association (AMIA), the professional society for health-related informatics, and bring words from our President and CEO, Dr. Kevin Fickenscher, who unfortunately could not be here today.

Today, AMIA recognizes the tremendous accomplishment of the CDC’s Scientific Education and Professional Development Program Office in their collaboration with the Department of Labor. The establishment of CDC’s Public Health Informatics Fellowship Program (PHIFP) as a Department of Labor (DOL) Registered Apprenticeship and the laying of the foundation for a standard occupation code for public health informaticians represent a very significant milestone in the decade-long effort by AMIA and its members and leaders to address the crucial issues of informatics workforce development.

The mere fact that the Department of Labor may soon give informaticians a workforce code is very encouraging.  The designation will help the nation realize its ambitions for supporting a fully-interoperable, data-driven learning healthcare system.  Key to this ambition are informaticians of all stripes, not only public health but also clinical, nursing, and even bioinformatics.

In 2001, I and more than 400 AMIA member experts and thought leaders gathered for the organization’s Spring Congress meeting here in Atlanta to develop a national agenda for public health informatics. The resulting 74 recommendations emerged with themes reflected in the CDC/DOL’s decision. Our stakeholders recognized the need to be engaged in coordinated activities related to public health information. They also forecasted the need for informatics training throughout the public health workforce.

A decade later, AMIA experts revisited the national agenda at the 2011 Spring Congress meeting, where we came up with recommendations supporting the need for informatics workforce development and underlining informatics crucial role in the future of public health and healthcare.

Many of us in AMIA are involved in complementary efforts in workforce development in other areas of informatics. I myself have had the opportunity over the last two years to play key roles in the health IT workforce development programs of the Office of the National Coordinator for Health IT, both training professionals in clinical informatics as well as developing the national health IT curriculum focused initially on community college programs but now freely available to the entire world.

Public health informatics is embedded in these efforts, as those in clinical informatics must comprehend how the public health system can benefit from our federal investment in adoption and meaningful use of electronic health records. This is exemplified at my institution, OHSU, where a CDC public health fellowship graduate serves on our faculty and teaches a course in public health informatics to a predominantly clinical informatics student body.

AMIA members will continue to lead the national discussion on informatics workforce development and on what is needed on the front lines of public health. Together with leadership from the federal government, NGOs, public health organizations, associations and specialty societies and business we know informatics professions will grow.  We are encouraged that the CDC/DOL’s ‘public health informaticians’ designation can open the door for other informatics-related designations to follow.

Informatics Professor Elsewhere on the Web

I have had the opportunity to have my blog-related work featured elsewhere on the Web. Some of these sites get more traffic than my own blog.

One site where I have been having edited versions of my blog posts re-posted is HITECH Answers. All of the postings can be found by searching on the tag assigned to them indicating they are from me. HITECH Answers also features a radio show called MULive, where I was the guest on April 3, 2012. (The audio archive of the show can be accessed by registering or going straight to audio link.)

Another site re-posting some of my blog entries of pertinence to internal medicine physicians is the American College of Physicians. Some of my postings are available on ACP Internist, aimed at all internists, while others are available on ACP Hospitalist, aimed at hospitalist physicians.

Postscript: Shortly after this entry was posted, Dr. Kevin Fickenscher, new President and CEO of AMIA, called out a recent posting of mine.

From Implementation to Analytics: The Future Work of Informatics

I am occasionally asked whether the work of informatics will be "done" when everyone is finishing implementing electronic health record (EHR) systems. Sometimes the query is further qualified by, "once everyone gets their HITECH money."

My answer is always an emphatic "No!" There is no question that some informatics implementation activity may slow down when healthcare organizations are no longer fueled by pursuit of HITECH incentive dollars. These activities may be impacted even further by bottom line woes that are likely to impact healthcare no matter what the outcome of healthcare reform, or whatever other distractions come along, such as ICD-10.

I often further qualify my answer by noting that for many of us, the real interesting work of informatics begins when the EHR platform is in place and we can truly start to do interesting things with the data. These are the so-called "secondary uses" or "reuses" of clinical data [1], things like quality measurement and improvement, improved clinical research, or indeed the "learning health system" first envisioned by the Institute of Medicine [2] and put in the context of the HITECH investment by Friedman et al. [3]. Some call this the "optimization" stage of EHR implementation [4].

One buzzword that is used increasingly in healthcare (and was already in use outside of healthcare over the last few years) is analytics. As with all buzzwords, there is a copious volume of material that has been written. I find a couple books by Tom Davenport and associates [5, 6] to provide good overviews. Davenport is Research Director for a company in Portland called the International Institute for Analytics. A recent primer by The Advisory Board Company, a healthcare consulting firm, gives a good overview of analytics in the context of healthcare [7].  Another recent report comes from PwC, which paints a similar picture of the near future, although (to my content!) describes this as clinical informatics (rather than analytics) [8], The phrase business intelligence is sometimes used to describe this work, and I suspect we will see another phrase, big data, appearing more frequently, especially with the recent Obama Administration initiative in this area [9].

The Advisory Board Company primer nicely paints an overview of the use of analytics and business intelligence in healthcare. They distinguish between different uses of the data, each requiring a higher level of analysis and complexity:

• Descriptive - reporting and querying of data to identify problems and solutions
• Predictive - modeling, forecasting, and simulating outcomes based on the data
• Prescriptive - recommend the best course of action based on the data

Of course, those of us who work in clinical informatics know that gleaning value from clinical data is challenging. Indeed, those who have learned from implementation in the trenches may be best qualified to understand the limitations of their data. As I often say, documentation is not usually the highest priority for busy clinicians. Indeed, it is often what stands between a tired clinician at the end of the day and being able to go home for dinner. Clinical data also suffers from the lack of standards in structure and terminology of data, and it is often fragmented across different systems, both within and across different healthcare organizations.

Nonetheless, the growing platform of electronic clinical data, fueled initially by EHR adoption and now augmented by efforts at health information exchange in the proposed rules for Stage 2 of meaningful use, point the way forward [10]. Regardless of one's political views of healthcare reform, it is clear that the system needs to change to become more accountable and efficient. This will be drawn out with the move to new delivery systems, such as accountable care organizations [11]. Thus, analytics and related activities are the future of clinical informatics, realizing the goal of my definition of the field, which is the use of information to improve individual health, healthcare, public health, and biomedical research [12].

References

[1] Safran, C., Bloomrosen, M., et al. (2007). Toward a national framework for the secondary use of health data: an American Medical Informatics Association white paper. Journal of the American Medical Informatics Association, 14: 1-9.

[2] Olsen, L., Aisner, D., et al., eds. (2007). The Learning Healthcare System - Workshop Summary. Washington, DC. National Academies Press.

[3] Friedman, C., Wong, A., et al. (2010). Achieving a nationwide learning health system. Science Translational Medicine, 2(57): 57cm29. http://stm.sciencemag.org/content/2/57/57cm29.full.

[4] Walker, J., Richards, F., et al., eds. (2006). Implementing an Electronic Health Record System New York, NY. Springer.

[5] Davenport, T. and Harris, J. (2007). Competing on Analytics : The New Science of Winning. Cambridge, MA. Harvard Business School Press.

[6] Davenport, T., Harris, J., et al. (2010). Analytics at Work: Smarter Decisions, Better Results. Cambridge, MA. Harvard Business Review Press.

[7] Adams, J. and Klein, J. (2011). Business Intelligence and Analytics in Health Care - A Primer. Washington, DC, The Advisory Board Company. http://www.advisory.com/Research/IT-Strategy-Council/Research-Notes/2011/Business-Intelligence-and-Analytics-in-Health-Care.

[8] Anonymous (2012). Needles in a haystack: Seeking knowledge with clinical informatics, PriceWaterhouseCoopers. http://www.pwc.com/us/en/health-industries/publications/needles-in-a-haystack.jhtml.

[9] Anonymous (2012). Obama Administration Unveils “Big Data” Initiative: Announces $200 Million in New R&D Investments. Washington, DC, White House. http://www.whitehouse.gov/sites/default/files/microsites/ostp/big_data_press_release_final_2.pdf.

[10] Copoulos, M., Raiford, R., et al. (2012). The Next Chapter - First Look at the Proposed Rule on Stage 2 of Meaningful Use. Washington, DC, The Advisory Board Company. http://www.advisory.com/Research/IT-Strategy-Council/Research-Notes/2012/~/media/Advisory-com/Research/ITSC/Research-Notes/2012/The-Next-Chapter-Stage-2.pdf.

[11] Fisher, E., McClellan, M., et al. (2011). Building the path to accountable care. New England Journal of Medicine, 365: 2445-2447.

[12] Hersh, W. (2009). A stimulus to define informatics and health information technology. BMC Medical Informatics & Decision Making, 9: 24. http://www.biomedcentral.com/1472-6947/9/24/.

Informatics Evidence, Redux

About a year ago there was a big dustup in the informatics field concerning a study published by Romano and Stafford in Archives of Internal Medicine that purported to show that electronic health record (EHR) use was not associated with improved quality of care [1]. As honest informaticians, we need to take such research seriously, aiming to improve what we do based on the evidence. This study, however, was problematic, in that it was based on an older data set not designed for answering questions such as the one asked by Romano and Stafford. A better approach would have been to perform a prospective clinical trial that directly assessed an informatics intervention, one of which was indeed published a few months later that did show improvement in care augmented by use of an EHR [2].

Now comes a similar situation a year later with the publication of a study by McCormick et al. in Health Affairs, which uses the same data source to show that physicians who have access to computerized imaging results (not necessarily via an EHR) have a 40-70% higher likelihood of ordering imaging tests [3]. This study set off a similar conversation about whether we are jumping the gun, especially with regards to the substantial federal investment in EHR adoption through the Health Information for Clinical and Economic Health (HITECH) Act. As with the Romano and Stafford study, this new study set off a lot of debate, including an exchange between the National Coordinator for Health IT and a rebuttal by the authors.

It is unfortunate to have to reiterate that we should be guided by the evidence, but given that many of us do have careers staked on the success of the HITECH Act, we must acknowledge potential biases and be as objective as possible in evaluating research results. That said, the study by McCormick truly uses a very weak methodology and certainly does not justify the sweeping conclusions by the authors in their paper or their rebuttal.

Similar to the Romano and Stafford study, this study makes associations with data sources that are not really designed to answer the question of whether EHRs will reduce test ordering. Of course, an even more fundamental question is whether reduced test ordering is something we desire anyways. While imaging tests are clearly over-utilized in healthcare [4,5], this study is incapable of telling us the value of the imaging that was ordered in increased amounts by physicians with access to electronic results. As such, we have no clue as to whether the imaging may or may not be warranted, or how the increased ordering impacted care of the patients for whom it was ordered. It is entirely possible that the increased imaging was beneficial in the management of those patients.

The authors conclude at the end of their abstract that "use of these health information technologies, whatever their other benefits, remains unproven as an effective cost-control strategy with respect to reducing the ordering of unnecessary tests." This is indeed a broad, sweeping conclusion that is hardly warranted from the methods or results of their study.

The rebuttal by Dr. Mostashari is reasonable, noting that the results of this study tell us little of the value of EHRs and the HITECH investment, which is what the authors seem to want to criticize in their results. This certainly comes out in their rebuttal, where they use a number of adjectives to impugn his motives. I do agree with their call for dialogue as well as well-designed clinical trials to assess the benefits of health IT. It may well be that the results of their research are true, and that EHRs will increase costs by making the ordering of expensive tests easier. But we really need our research to answer the larger questions of the value to patient outcomes. We also need to explore how larger changes in our healthcare system, particularly a reimbursement approach that favors quantity over quality, will be empowered by better information systems. With research focusing on those questions, we will be able to ascertain the true value of health IT and how we can improve our use of it.

References

[1] Romano, M. and Stafford, R. (2011). Electronic health records and clinical decision support systems: impact on national ambulatory care quality. Archives of Internal Medicine, 171: 897-903.

[2] Cebul, R., Love, T., et al. (2011). Electronic health records and quality of diabetes care. New England Journal of Medicine, 365: 825-833.

[3] McCormick, D., Bor, D., et al. (2012). Giving office-based physicians electronic access to patients' prior imaging and lab results did not deter ordering of tests. Health Affairs, 31: 488-496.

[4] Baker, L., Atlas, S., et al. (2009). Expanded use of imaging technology and the challenge of measuring value. Health Affairs, 27: 1467-1478.

[5] Hillman, B. and Goldsmith, J. (2010). The uncritical use of high-tech medical imaging. New England Journal of Medicine, 363: 4-6.

Update on the ONC Health IT Curriculum Project

It has been a while since I provided an update of the Office of the National Coordinator for Health Information Technology (ONC) Health IT Curriculum Project. I had the opportunity to give a presentation about the curriculum at this week's HIMSS Conference, so will use the preparation for that to give an update here.

The major news from the project is that the third version of the curriculum will be released in the next month. Version 3 will have the same component names and structure, but the content has been substantially revised and improved. In addition, there will be much more consistency of the slide formats as well as file content and naming. The content itself has been revised based on feedback obtained by a variety of mechanisms, including contracting with the American Medical Informatics Association (AMIA) and expertise they garnered in a process last summer. The materials also have improved accessibility for those with disabilities.

Some have expressed some concern that the project "ends" on April 2, 2012. While it is true that the ONC grant ends on that date, the Web site will continue to be available beyond then. ONC is also considering a no-cost extension of the grants. Stay tuned for more details.

As with Version 2, the Version 3 materials will be made available to the general public. Anyone will be able to go to the Web site of the National Training & Dissemination Center (NTDC) Web site and create a login to enable downloading of the materials.

Other news includes a mention of the curriculum as one of the major accomplishments of ONC for 2011, according to the National Coordinator, Dr. Farzad Mostashsari.

Another useful accomplishment was the addition of a search capability to the NTDC Web site. The search engine allows searching over all text-containing documents. The search engine output allows list the files containing the search terms and allows downloading of the individual file or the unit .zip file that contains the file. The search engine indexed the 1342 Word documents and 460 Powerpoint files and has made them available for word-based searching. (For language trivia buffs, there are 37,485 unique words in these files.)

Additional news about the project includes data about the size of the Version 2 materials as well as download data since its release, including public users.

The entire collection of materials, including the slides, voice-over narration of the slides, and other materials, is 7.84 gigabytes in size. There are a total of 33,172 files. This actually does not include the VA VistA for Education electronic health record system, which has an installer file that is another 770 megabytes in size. VistA requires a license for the Intersystems Cache system, which is freely available to academic institutions but not others. The narration of the slides, available as both Flash-based "video" as well as MP3 audio files, totals 125.6 hours. As noted above, the materials have 1342 Word documents and 460 Powerpoint files. The latter contain a total of 8913 slides.

We also have details about the downloading of Version 2, covering the period from the public rollout to the end of 2011, about one-half year. Before delving into detail about the downloads, it is important to remember the structure and contents of the curriculum. The curriculum consists of 20 components, each aiming to be comparable in size to a three-credit college course. These courses are part of the various workforce roles around which the ONC community college workforce development program is organized, but of course can be used independently either as a whole course or even broken into parts. Each component is broken down into 8-12 units. Each unit contains voice-over-Powerpoint lectures (with transcripts), self-assessment quizzes, and other learning activities (such as discussions and hands-on exercises).

The NTDC web site is structured for downloading by units. The workforce for someone downloading is to create a login (or, in the case of community college faculty users, have a login created, which allows access to additional curricular support) and then navigate through the components to the individual units (packaged in .zip files) for downloading. (We do plan to implement the ability to download entire components in 2012.) Also available for downloading is .zip file containing all the component blueprints (syllabus-like documents) as well as the installer and a help file for VistA for Education.

All told, there were 284,398 downloads of Version 2 units and other files between May-June and the end of 2011. It is important to put this large number in context, which it represents the number of items downloaded. These downloads were carried out by 537 community college faculty and 4680 public users. The public users came from 31 different countries, although the vast majority were from the US. Many of the registering public users did not provide the information the system asked when creating the login, so their background and demographics are not accurately characterized, but browsing of the log shows many educators as well as individuals connected to health care organizations.

The components with the largest number of downloads were Components 3 (22,645), 5 (19,504), and 1 (18,920). The average number of unit downloads per component varied from 2102 for Component 1 to 931 for Component 13. The component blueprints file and VistA for Education installer were downloaded 3255 and 3136 times respectively.

Additional insight can be gained from looking at the minimum and maximum amount of downloads of units within each component. This provides a sense of how many users are downloading one or more units within a component. The minimum number of units downloaded within a component tend to be much closer across the components than the total number of downloads or the maximum. For example, one of the units of Component 5 was only downloaded 925 times, which was not much more than the most minimally downloaded unit of Component 20 (923). This implies that there might be two downloader types: those who take everything for a given component and those who pick and choose.

All told, we are pleased that the ONC Health IT Curriculum has become a substantial global resource. It will be improved with Version 3 that is coming shortly. We are also exploring ways to sustain it beyond the end of the HITECH funding.

eHealth Initiative Report on Hiring in Health Information Exchanges

The eHealth Initiative (eHI), a health information exchange (HIE) advocacy group, recently released a report stating that while HIEs are likely to generate jobs in health information technology (HIT), few of those jobs have gone to those trained by the workforce development programs of the Office of the National Coordinator for Health Information Technology (ONC). As one who is associated with the ONC workforce development programs, I was naturally alerted to the report as well as an article in the trade publication, Health Data Management.

My overall reaction to the eHI report is that while I do not disagree what its findings and conclusions, I do believe those findings and conclusions need to be viewed as part of a larger perspective about HIE and HIT employment. I also believe that the reporting of the methodology used for the report is incomplete, leading to some uncertainty about the meaning of its findings and conclusions. In particular, I wonder whether the report authors or those surveyed fully understand the ONC workforce program or even the HIT workforce itself. There may be more information about this report in one of eHI's proprietary publications, but I cannot find anything on their Web site. (Although I support the work of eHI, I am not a member.)

I do acknowledge up front that I have a vested interest in the ONC workforce development program. I am funded on two grants, one devoted to curriculum development for the six-month community college programs and another for university-based training in the Oregon Health & Science University (OHSU) graduate program in biomedical informatics. I also believe it is fair for anyone to question the value of these programs and whether the investment being made is productive.

My main problem with the report is that its methodology is incompletely described. To begin with, the report itself has no listed author(s) or contact information. Who carried out this report and how can they be contacted?

A related concern is whether those who developed the report's survey or those who answered it are sufficiently knowledgeable about the ONC workforce program itself.  The report does not describe how the question(s) about the ONC programs was/were asked or how knowledgeable the respondents were about the different programs. Many people, for example, are unaware that the program is larger than just the six-month community college certificate programs. Are they knowledgeable, for example, of the university-based training (UBT) programs, which have a workforce role called "Health Informatics Management & Exchange Specialist" that is likely to be most amenable to work in HIEs? This workforce role has been the most subscribed workforce role among the six covered by the UBT programs.

By the same token, the report does not put employment within HIEs in perspective. While I certainly believe that HIEs are a critical element to the larger success of HIT adoption, it is important to remember that the number of people employed in HIEs will be a relatively small part of the overall HIT workforce. For any given state or region, there are many healthcare organizations whose HIT systems will feed into one or a small numbers of HIEs. Although there will be many important jobs for those who implement, lead, and utilize HIEs, their numbers will be modest relative to the large number of HIT professionals in hospitals, physician offices, and other health-related organizations. It is just simple math.

Also important to remember is that many HIEs are still early in development, where the critical skills are more around planning and development than implementation. It is not surprising to see consultants being heavily used, as opposed to professionals just out of their education programs without a great deal of workforce experience. Related to this, the report seems to look only at direct hiring of ONC workforce program graduates by HIEs. We do not know how many graduates of ONC programs work for consultants, vendors, or others as opposed to being hired directly by HIEs, on which the report seems to focus

We also need to remember that HIE jobs vary in the same way that HIT and clinical informatics jobs do. As such, those trained in the ONC programs might not be a fit for the jobs available in HIE. In fact, it is likely that HIE jobs require a great deal of HIT workforce experience, which ONC workforce program graduates by definition do not have.

The Health Data Management article also goes off on a tangent and raises some issues about the ONC HIT curriculum. Some of these are valid criticisms, but it is also important to remember that these curricular materials are designed for HIT teachers, who are encouraged to use them creatively to offer a meaningful learning experience and not just a rote curriculum out of the box. Some of the community colleges have done this better than others in this regard.

While I applaud eHI for brining the workforce issue in the context of HIE to light, I also believe that their report raises more questions than it answers. I do hope that someone will come forth and explain the details of the report's methodology and its findings. I will certainly make a postscript to this entry in my blog if anyone does so. I also encourage dialogue about the value of the ONC workforce programs and how we can improve them not only for content, but also the employability of their graduates.