National Library of Medicine: An Informatics and Government Agency Exemplar

This week I am off to another meeting I attend every year, which is the National Library of Medicine (NLM) Informatics Training Conference, the annual meeting held for all trainees funded under the NLM Biomedical Informatics Training Grant Program. Also in attendance are program directors and faculty, NLM staff, VA informatics trainees, and a variety of other people. The meeting varies between being held at the NLM and the various sites; OHSU hosted the meeting in 2009.

At a time when Americans increasingly question the function and value of their government and its agencies,  the NLM is a shining testament to the good that the public sector can perform. It is hard to imagine a private entity carrying out the mission of NLM, especially as successfully as it has done so.

The NLM is the world's medical librarian, providing an entry way into the biomedical literature for anyone on the planet who types pubmed.gov into a browser. (The Pubmed system provides access to the MEDLINE bibliographic database, which contains the title, abstract, source information, and other metadata about scientific journals articles in biomedicine.) Even though most of the articles referenced in MEDLINE are from commercial publishers and not freely accessible, NLM delivers users to the publishers' electronic doorsteps. The NLM and its talented scientists and developers have pushed the envelope in many other areas as well, from genomics to imaging to public health. The NLM serves not only researchers and clinicians, but also consumers and policy makers.

Another critical role of the NLM is its scientific leadership in the field of biomedical and health informatics. The NLM funds research in informatics as well as the training of future scientists and leaders. While not the only federal agency involved in the use of information technology in health and biomedicine, it is clearly the foundational leader that facilitates the basic research to inform others who apply it.

No small part of the NLM's success is due to its excellent leadership in Donald AB Lindberg, MD, who has guided the Library for over two decades, longer than I and many others have been in the field. Dr. Lindberg has been remarkably prescient over the years. I remember him touting the virtues of the Human Genome Project when I was an NLM informatics trainee in the late 1980s. Subsequently he has been spot on in his seeing the development of new venues for publishing as well as the desire for patients and consumers to access health information online.

The NLM also has longevity. It has an illustrious history, dating back to its inception as the The Library of the Office of the Surgeon General of the Army, led in its early days by John Shaw Billings, MD. This year is the NLM's 175th year anniversary.

I have a great deal of gratitude for the NLM personally. Like many who work in informatics, my career would not be what it is without the help of NLM. I entered the field in a postdoctoral fellowship directly out of my medical training in 1987. The three years of fellowship funded by NLM allowed me to gain knowledge and skills as well as prepare for an academic career in the field. After completing my informatics training, I landed a faculty position at Oregon Health & Science University (OHSU), funded by a grant to OHSU under the Integrated Advanced Information Management Systems (IAIMS) program, an NLM initiative to develop the informatics human and technology infrastructure at academic medical centers. (In the 21st century, these activities are a normal part of doing business at academic medical centers.) The director of the OHSU IAIMS program, who recruited me to that first job, J. Robert Beck, MD, also obtained an NLM informatics training grant at OHSU, of which I now serve as PI and Director.

The NLM has also funded my research over the years, not only providing the resources for my own scientific contributions to the field but also giving me the experience and latitude to develop other aspects of my career. My first grant ever was a First Independent Research Support & Transition (FIRST) Award (also known as an R29). Since then I have had a number of subsequent grants both for research and education of trainees. These projects, from research to teaching, have enabled me to touch the life of countless others who have also achieved success in their careers in the field.

While it is obvious that the US government needs to make some painful decisions about long-term debt control, discretionary expenditures such as those on NLM have been beneficial to many people, not to mention the health of Americans and others around the world. When politicians and policy makers are deliberating, I hope they will consider the value and impact that government agencies like the NLM have made to so many people. I will always be grateful for what the NLM has done for me.

Public Rollout of the ONC Health IT Curriculum

This week was a major milestone for the Office of the National Coordinator for Health IT (ONC) Health IT Curriculum project. The curricular materials that were developed for the 82 community college programs to rapidly expand the health IT workforce were released to all educators and the public at large. In this posting, I will provide the context for this project and describe what it is not before delving into the details of what the curriculum contains.

The ONC Health IT Curriculum is one of four programs in the overall ONC Workforce Development Program. The overall program was specified by Section 3016 of the Health Information Technology for Economic and Clinical Health (HITECH) Act, the portion of the American Recovery and Reinvestment Act (ARRA), also known as the federal stimulus bill. ONC operationalized the program by designating 12 workforce roles, with six to be educated in the six-month community college programs and six to be educated in 1-2 year programs in universities. The primary audience for the curricular materials are the community college programs.

Five universities were funded under the $10 million project as Curriculum Development Centers: Oregon Health & Science University (OHSU), Columbia University, Duke University, Johns Hopkins University, and University of Alabama-Birmingham. Each center prepared four components each. One university, OHSU, was additionally funded to serve as the National Training & Dissemination Center (NTDC), given the additional tasks of developing the dissemination Web site, training community college faculty in use of the materials, capturing feedback, and providing technical support. The curricular materials are now available for download by the public on the NTDC Web site, although the feedback and support functions are limited to the 82 community colleges.

The curricular materials are not a certificate or degree program out of the box. Rather, the content should be thought of more like a library (or, to use the words of ONC Chief Science Officer Charles Friedman, PhD, a "buffet") from which educators can pick and choose content for their courses. The materials alone will not substitute for formal education, as good education still requires teachers, mentors, and fellow learners with whom to interact (whether in-person or on-line). However, the matierlals will be a valuable resource for a wide variety of educational activities in health IT. As the director of a graduate program in biomedical informatics, I know that OHSU will adopt some of these materials in its own graduate-level educational program (just as some of the curricular content came from our existing program).

The curricular materials consist of 20 components, each of which is comparable in depth to a college course. The components are subdivided into 8-12 units, each of which contain a variety of activities appropriate to the topic, including voice-over-Powerpoint narrated lectures, references, suggested readings, exercises, and more. The topic areas of the components are: 

1. Introduction to Health Care and Public Health in the U.S.
2. The Culture of Health Care
3. Terminology in Health Care and Public Health Settings
4. Introduction to Information and Computer Science
5. History of Health Information Technology in the U.S.
6. Health Management Information Systems
7. Working with Health IT Systems
8. Installation and Maintenance of Health IT Systems
9. Networking and Health Information Exchange
10. Fundamentals of Health Workflow Process Analysis & Redesign
11. Configuring EHRs
12. Quality Improvement
13. Public Health IT
14. Special Topics Course on Vendor-Specific Systems
15. Usability and Human Factors
16. Professionalism/Customer Service in the Health Environment
17. Working in Teams
18. Planning, Management and Leadership for Health IT
19. Introduction to Project Management
20. Training and Instructional Design

ONC and the Curriculum Development Centers also developed a “set table” consisting of a matrix of curriculum components and workforce roles to guide community college programs in using components to train for particular workforce roles. The matrix specified the core set of components for each workforce role for two types of student backgrounds, healthcare and information technology.
http

Each component also contains a blueprint document that provides an overview of the learning objectives and content for each unit. All of the components also have an instructor's manual that provides more detailed information, including listing of authorship and teaching information. The full set of blueprints have been rolled into a single PDF portfolio and are available on the ONC Web site.

Three of the components are "lab" components that make use of an educational version of the Veteran's Administration (VA) VistA EHR. A version of VistA that runs under various versions of Microsoft Windows is provided on the Web site, courtesy of the VA. However, this version will not be usable by everyone, as it requires a license for the Intersystems Cache environment, which is freely available to academic institutions but not others. Nonetheless, the materials will still be valuable to others who can adapt the exercises for other EHR systems.

All told, the curricular materials are a comprehensive resource. The entire collection of material is 7.5 gigabytes in size (6.75 gigabytes compressed) in 12,339 files. The 20 components contain 213 units, 460 lectures (some units have more than one lecture), 8913 slides, and 125.9 hours of lecture audio. In the collection are 460 Powerpoint files, 460 MP3 files, 465 PDF files, and 1346 Microsoft Word files.

We call this publicly available version of the curricular materials Version 2. It has been available to the ONC Community College Consortium for two months, and supersedes the original Version 1 provided to consortium members last year. The materials are distributed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License. This means that all users of the curriculum can use, share, and adapt the materials but must attribute the originator of work, use the materials only for non-commercial purposes, and share any changes made under same license. Per the ONC, universities own the intellectual property for their components.

The support for the public rollout of the curricular materials will be minimal. This is in part because the funding does not have the resources to provide that support but also because these materials are aimed at educators who will adapt them into their own courses and other educational activities.

Another program in the ONC Workforce Development Program related to the project is the Competency Examination, a project led by Northern Virginia Community College. There are six exams, with one for each of the six community college-trained workforce roles. Each exam consists of 125 multiple-choice questions, to be taken in three hours and graded on a pass-fail basis. At least 80% of exam questions come from the curriculum components. Beta versions of the six exams became available on May 20, 2011, with the final versions to be ready in September. The exam is free to consortia member college graduates through their schools.

The final program in the is the University-Based Training (UBT) program, which funds training in the other six workforce roles deemed to require longer training at the university level. Nine universities or consortia thereof, including OHSU, were funded under this program. As noted elsewhere, OHSU recently had its first graduates from its UBT program, with many more in the pipeline.

The Curriculum Development Centers and ONC do not plan to rest on our laurels. We know there are many areas where Version 2 can be improved, and fortunately the two-year project includes additional funding to provide for a Version 3 that will be delivered in 2012. A planning process is underway to improve the content and technical aspects of Version 2, along with reducing its gap and overlaps.

It has been gratifying to be part of this project, which has consumed a great deal of my life since the project began in April, 2010. I have enjoyed all of the roles I have played, as Director of the NTDC, Director of the OHSU Curriculum Development Center, and author of several units. I will look forward to feedback about Version 2 and suggestions for enhancements in Version 3. How to sustain the curriculum once the ONC funding ends is also a key concern.

Informatics Destination: Buenos Aires

I spent part of last week with my friends and colleagues at Hospital Italiano de Buenos Aires (HIBA) in Argentina. The HIBA Department of Health Informatics is truly an international leader in the field, with an internally developed electronic health record (EHR) that serves the needs of the hospital's clinicians, patients, and researchers. HIBA is a large academic medical center in the heart of Buenos Aires and also has a large health maintenance organization (HMO), Plan de Salud, that serves nearly half a million people. It also has a young but growing university.

The HIBA EHR has been in development for over a decade. At a time when the "conventional wisdom" of informatics is to acquire and implement commercial systems, HIBA has built a system tailored to its organization and workflow. Their success is a testament to the vision and leadership of the program's founder, Fernan Gonzalez Bernaldo de Quiros, MD. Dr. Quiros started HIBA's Department of Medical Informatics a decade ago to provide leadership in developing and implement the system, called ITALICA. He has now assumed the role of Vice President for Strategic Planning of HIBA, while Daniel Luna, MD has stepped in to head the department. Now called the Department of Health Informatics, they oversee all aspects of IT at HIBA, including non-clinical applications. An excellent overview of all their work is provided in a Yearbook of Medical Informatics 2009 article: Quiros, F., Luna, D., et al. (2009). Experience in the Development of an In-house Health Information System and the Training Needs of the Human Resources at the Hospital Italiano de Buenos Aires, 147-152, in Geissbuhler, A. and Kulikowski, C., eds. IMIA Yearbook of Medical Informatics 2009. Stuttgart, Germany. Schattauer.

HIBA has also become an international leader in informatics education and training. When the department was established, they also launched a medical informatics residency program. This program has trained the human resources necessary for the success of ITALICA. An emerging leader in the educational program has been Paula Otero, MD.

I first met Dr. Otero in 2004. A year later, she enrolled in the very first offering of the OHSU-AMIA 10x10 course. After the course ended, she proposed to translate the course into Spanish to make it available to a Latin America audience. She and her team successfully translated the course and began offering it across Latin America. While the first version was mostly a direct translation, the course has since diverged from the US-based course to be more specific to health care in Latin America. (For example, very little HIPAA!) For more information, see: Otero, P., Hersh, W., et al. (2010). A medical informatics distance-learning course for Latin America - translation, implementation and evaluation. Methods of Information in Medicine, 49: 310-315.

This initial collaboration set the stage for other collaborative activities. Dr. Otero, Dr. Quiros, and I were involved in the Rockefeller Foundation workshop devoted to building human capacity in health informatics in the developing world in Bellagio, Italy in 2008. We subsequently worked together on the AMIA Global Partnership Program. Dr. Otero has become my Co-Chair in leading the International Medical Informatics Association (IMIA) Working Group on Education.

The crowning achievement of our collaboration was the awarding of a grant from Fogarty International Center of the US National Institutes of Health (NIH). In 2009, we were awarded one of eight grants in Fogarty's Informatics Training for Global Health (ITGH) Program. The stated goal of our project under this funding was to extend our collaboration that had mostly been in clinical informatics into clinical research informatics. HIBA has a strong Institute of Basic Sciences and Experimental Medicine, which includes 31 basic research teams. Many are funded by grants, including some from the NIH.

We proposed in the grant, and have operationalized in the first two-plus years of the project, a plan for short-term, intermediate-term, and long-term training. The short-term training has been focused on clinical researchers, extending the Spanish 10x10 course with modules that teach them how informatics can augment clinical research.

The intermediate training has been more focused on informatics trainees, with a course in clinical research informatics developed by OHSU informatics faculty Judith Logan, MD, MS. This course was taught on-line in OHSU's spring academic quarter to both OHSU and HIBA informatics trainees. Dr. Logan also came on this trip to have an in-person meeting with the HIBA students.

The long-term training has focused on providing postdoctoral fellowship training to HIBA informaticians. At OHSU, we have treated these trainees as if they were fellows on our National Library of Medicine (NLM) training grant. The first two fellows - Damian Borbolla, MD and Vanina Taliercio, MD - have been at OHSU for over a year. A third fellow, Sonia Benitez, MD, will join them later this year. The goal for these trainees is for them to return to Argentina after their training to assume leadership roles in informatics and clinical research.

Dr. Logan and I also had the opportunity to give talks at HIBA (with more details and even an Elluminate recording of the slides and audio). Not only were there about 80 people present in person, another 25 or so listened in via Webcast. Some of the Webcast listeners even asked questions of the speakers. In my talk I provided an overview of the HITECH program for EHR adoption in the US. Dr. Otero translated my slides to Spanish and both the English and Spanish versions, with references, are available on my Web site. An interesting piece of trivia I learned on this trip is that the phrase meaningful use has no direct translation in Spanish. The closest translation is uso significativo. (Which is somewhat ironic, since HIBA is much closer to meaningful use of EHRs than most US hospitals!)

Although we have made substantial progress in our collaboration, the best is yet to come. We will look forward not to our trainees applying their new knowledge and skills to advancing healthcare and clinical research in Argentina, but also to new undertakings, such as a possible jointly developed master's degree.

Commencement Address Representing OHSU School of Medicine Graduate Studies Program

The Commencement & Hooding Ceremony of the OHSU School of Medicine, also known as Graduation, is always an enjoyable time for me. It is gratifying to see another year's class of graduates from our Biomedical Informatics Graduate Program receive their hoods and diplomas. This year we had largest graduating class ever, with six PhDs, 12 students in our two master's programs, and 20 Graduate Certificates. This brings our total number of degrees and certificates awarded up to 336 since the inception of the program in 1996. This commencement we more than doubled our number of PhD graduates (from five to 11), and also saw the first group of graduates from our ONC University-Based Training (UBT) Program.

This year I also was invited to give the Graduate Studies Program faculty address. I was honored to represent the faculty of all of the OHSU School of Medicine graduate programs and share my informatics-tinged wisdom and vision with the larger School of Medicine audience.

Below is the text of my remarks delivered on Monday, June 6, 2011:

After all these years of sitting down in the faculty section of this ceremony, I am honored to be asked to give this address representing the faculty in the graduate programs of the School of Medicine. As some of you know, I direct the graduate program in biomedical informatics, which is the field devoted to the use of data and information, usually but not always aided by computers, to improve personal health, clinical practice, biomedical research, and public health. Like all disciplines, biomedical informatics has a science and methodology that is carried out by its researchers and practitioners, and a new group of graduates are entering the field by completing their studies today.

As the faculty in my program know, Commencement is a very important event for me. With the exception of last year due to an unavoidable conflict, I have attended every Commencement since our biomedical informatics graduate program had its first graduates in 1998. We began with a handful of Master's degrees, but now as of this graduation have over 300 alumni who have attained not only Master's degrees, but also PhDs and Graduate Certificates. Despite 13 years of graduating students, my thrill of seeing graduates of our program has not worn off. I am sure that my fellow graduate program directors feel the same way.

So what advice can I give to those who are graduating with PhDs, Master's degrees, and Certificates in the School of Medicine? I will skip the usual advice, important as it is, to devote your life's work to your profession, to keep a healthy balance of activities outside of work with family and friends, and to act professionally in a world of instant gratification and 24/7 information flow. Instead, I will try to provide some perspective and wisdom from my discipline of biomedical informatics.

I probably do not need to tell graduates, faculty, or even members of the audience that the 21st century is a golden era at the intersection of health sciences with information and computer sciences. It is truly changing what we do as clinicians, researchers, and other professionals who deal with health.

One of the best statements of this vision comes the Institute of Medicine and is the notion of the learning health system. We now truly have the ability to track and measure what we do in health care practice and public health, and drive research questions and answers from it. Our substantial federal investment in electronic health records, along with the growing ability to sequence genes, measure their expression, and analyze the products they produce, is ushering in an unprecedented era to compare and then learn the best approaches not only to treating disease but also keeping us healthy.

It is also critical to remember that no matter from what discipline you are graduating, success in this new era will require skills to use and manage information in ways that did not exist even a decade ago. You must understand the meaning and the limitations that exist with the increasing types and volume of data you collect. You must adhere to data standards so others can build on your work. Those of you working with human data also cannot forget the importance of protecting the privacy of individuals who have graciously permitted you to borrow their data for your work. In addition to skills in managing data, you must also be an expert in searching and accessing the literature and other scientific resources of your field. As if that is not enough, critical thinking and analysis are essential to all of this voluminous amount of data and information.

Another critical challenge to emerge in the 21st century is the need to collaborate across disciplines. The truly vexing problems of health care and public health require an interdisciplinary approach. Basic scientists, clinicians, informaticians, and others must come together to translate basic science into clinical care, to bring the best clinical care to the entire population, and insure that care is delivered with the highest quality and safety. We also need to reform our health care system to provide incentive for coordination and efficiency, not only because it will cost less but also because it will result in better patient outcomes. This will in turn require critical investments in information systems to bring the right information to the right people at the right time.

In closing, no matter what graduate degree or certificate you are receiving today, there are unprecedented opportunities. There may be uncertainties about health care reform, federal research funding, and the economy in general. But there is now unprecedented opportunity to impact health. I wish all graduates here today the best as they embark on their new careers.

(Postscript: The text of this talk also appears on the OHSU School of Medicine Commencement 2011 site.)

An Informatics Silver Lining to a Terrible Tragedy

Although the tornado in Joplin, Missouri was a terrible and unfortunate tragedy, there is an interesting little side story related to biomedical informatics. I don't want to make light of the tragedy, particularly the town having its hospital destroyed. However, an article on the St. Louis Today web site tells an interesting sidebar.

Apparently the destroyed hospital made its conversion to electronic health records (EHRs) just three weeks before the tornado. The EHR system did not miss a beat, and remained running during and after the storm. As such, people needing their records accessed were able to have that done when they obtained medical care elsewhere.

This situation brings memories of Hurricane Katrina, where just about all of the hospitals in New Orleans had their medical records rooms, typically in the basements of their facilities, destroyed by the ensuing flooding. The one exception was the New Orleans VA Medical Center, which was able to keep its records intact through the well-known VA EHR system.

Joplin also did have a health information security breach from the tornado. Although unlike most breaches we read about lately, this breach was purely due to non-electronic records, in particular paper records and x-ray films being blown up to 75 miles away.

This story does not alleviate the terrible tragedy of the tornado, nor does it rebut any of the serious challenges to implementing EHRs. It does, however, show one example of the value of electronic data systems in healthcare.

Update on the ONC Curriculum Development Centers Program

I recently posted an update about one of our Office of the National Coordinator for Health Information Technology (ONC) projects, the Oregon Health & Science University (OHSU) offering of the ONC University-Based Training (UBT) program, and promised an update to follow on our other grant, the Curriculum Development Centers program. The latter is a $10 million program for five universities – Columbia University, Duke University, Johns Hopkins University, Oregon Health & Science University (OHSU), and University of Alabama-Birmingham – to develop curricular materials for the 82 community colleges delivering short-term training for six of the 12 ONC-defined workforce roles. One university, OHSU, was provided additional funding to serve as the National Training & Dissemination Center (NTDC) that is additionally tasked with developing a Web site for dissemination of the materials, training community college faculty in their use, and capturing and distributing feedback collected from community college faculty.

As with the UBT program, the Curriculum Development Centers have been funded since April, 2010. Since that time, substantial progress has been made. The first version of the curriculum was delivered to the community colleges in two halves, one in August, 2010 and the other in October, 2010. Because of the tight timeline of the curriculum deliverables and the start-up of the community college programs, it was decided to not disseminate Version 1 beyond the five community college consortia overseeing the 82 member colleges. This also led to the decision for Version 2 to be delivered relatively quickly, in the spring of 2011, and mainly be an incremental update focused on improving the clarity and technical quality without making any major content overhaul. It was also decided that Version 2 would be the release promised in the original Request for Proposals (RFP) to be made available to all institutions of higher learning, which for all practical purposes means the general public. This public roll-out will take place in the summer of 2011.

Recall that the community college short-term training programs are focused on six of the 12 workforce roles that ONC has deemed necessary to help eligible professionals and hospitals achieve meaningful use of the electronic health record (EHR). (The other six workforce roles are trained by the UBT program.) Each of the 82 community colleges can offer certificates in one to six of the workforce roles, while the consortium to which it belongs must offer all six across their region.

The curriculum consists of 20 components, each of which is comparable to a college-level course (which of course can vary widely based on the length, depth of material, background of students, and other factors). The components are not called courses because it is up to the community colleges to turn them into actual courses in their programs. The colleges can use the materials “out of the box,” with little or no modification, or they may modify them as they desire for the needs of their programs.

ONC and the Curriculum Development Centers also developed a “set table” consisting of a matrix of curriculum components and workforce roles to guide community college programs in using components to train for particular workforce roles. The matrix specified the core set of components for each workforce role for two types of student backgrounds, healthcare and information technology.

Each component has a “blueprint,” which provides learning objectives and a detailed overview of the content. Each component is broken down into 8-15 units, which correspond roughly (though variably) to one week of a course. Each unit typically consists of learning objectives, a narrated slide lecture (delivered as Powerpoint slides, MP3 audio files, and narrated voice-over-Powerpoint Flash files), references, exercises, and other materials. (The blueprint for Version 1 on the ONC Web site will soon be replaced by the one for Version 2.)

The topic areas of the components are: 

1. Introduction to Health Care and Public Health in the U.S.
2. The Culture of Health Care
3. Terminology in Health Care and Public Health Settings
4. Introduction to Information and Computer Science
5. History of Health Information Technology in the U.S.
6. Health Management Information Systems
7. Working with Health IT Systems*
8. Installation and Maintenance of Health IT Systems*
9. Networking and Health Information Exchange
10. Fundamentals of Health Workflow Process Analysis & Redesign
11. Configuring EHRs*
12. Quality Improvement
13. Public Health IT
14. Special Topics Course on Vendor-Specific Systems
15. Usability and Human Factors
16. Professionalism/Customer Service in the Health Environment
17. Working in Teams
18. Planning, Management and Leadership for Health IT
19. Introduction to Project Management
20. Training and Instructional Design

Components 7, 8, and 11, denoted with an asterisk* above, are “lab” components that provide hands-on instruction. These components make use of a fully functioning version of the Veteran’s Administration VistA EHR that is included with the curricular materials and can be installed under most flavors of Windows (as well as some Windows virtual machines that run under MacOS and Linux).

Another program in the ONC Workforce Development Program related to the project is the Competency Examination, a project led by Northern Virginia Community College. There are six exams, with one for each of the six community college-trained workforce roles. Each exam consists of 125 multiple-choice questions, to be taken in three hours and graded on a pass-fail basis. At least 80% of exam questions come from the curriculum components. Beta versions of the six exams became available on May 20, 2011, with the final versions to be ready in September. The exam is free to consortia member college graduates through their schools.

As noted above, Version 2 will be released to all institutions of higher education in July, 2011. The details of how to access the materials will be provided at that time. For this release, the Curriculium Development Centers adopted a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License. This means all users of the curriculum can use, share, and adapt the materials but must attribute originator of work, use the materials only for non-commercial purposes, and share any changes made under same license. Per the original RFA, universities own the intellectual property for their components.

The Curriculum Development Centers have also started planning for the third and final version that will likely be released in early 2012. Planning for this version is underway. Unfortunately, there is not now any plans for continued funding, at least by ONC, beyond the project end in April, 2012. It is conceivable that some sort of open-source approach could be adopted to keep the curriculum going, but I do not see the resource continuing to be viable without some investment, at least in its infrastructure. Nonetheless, I am pleased overall with the project and I believe it will be an enduring contribution to the biomedical and health informatics community. I am looking forward to Version 3 and whatever opportunities there are to continue the project beyond it.

Physician Certification in Informatics

I am frequently asked about the status of certification of physicians in informatics. I did touch on this topic briefly in my posting on informatics opportunities for physicians last fall, but let me address the question of certification in more detail in this posting.

The motivation for physician certification in informatics is to recognize the growing stature and need for professional expertise of physicians who spend a significant amount of their time performing informatics-related duties. This includes not only the growing role of the Chief Medical Informatics Officer (CMIO), but other jobs where a physician draws on his or her expertise at the intersection of medicine and informatics.

The "gold standard" for any type of certification of physicians is board certification. There are currently 24 specialty boards (e.g., internal medicine, family medicine, pediatrics, surgery, radiology, preventive medicine, etc.), most of which have subspecialty boards as well (e.g., cardiology, hematology/oncology, and general internal medicine in internal medicine). Some subspecialties, such as geriatrics and palliative medicine, are offered by more than one specialty board. This will be the model for the clinical informatics subspecialty, and in fact the goal will be for it to be offered by all 24 specialty boards.

A comprehensive overview of the rationale and plan for developing the clinical informatics subspecialty was published in early 2010 by Detmer et al. [1]. This paper described the development of medical specialties and subspecialties generally and in the context of the new proposed subspecialty of clinical informatics. A more recent overview of the status board specialties was published last year and included mention of the proposed one for clinical informatics [2].

The proposal to establish the clinical informatics subspecialty was developed by the American Medical Informatics Association (AMIA) and submitted to the ABMS in 2010. The lead board submitting the proposal was the American Board of Preventive Medicine (ABPM), which has since been joined by the American Board of Pathology. The proposal had its first "reading" in early 2011, with a second reading and possible vote coming in September, 2011.

Certification in clinical informatics will work like any other multi-board subspecialty. To become certified, a physician will need to meet certain training requirements and then pass a certification exam. In the early years (usually the first five years of a specialty's existence), those with a certain level of experience will be able to "grandfather" in on the training requirements in a "practice track" and certify by passing the exam only. Those training after the initial practice track period will be required to complete some sort of fellowship in the specialty. The practice track requirements for clinical informatics will be determined after the ABMS approves the subspecialty and will likely apply to those with some defined level of time and depth of experience in clinical informatics settings.

If the ABMS proposal is approved, the ABPM will begin development of a certification exam, which will likely become available in the fall of 2012 for those meeting the practice track requirements. The next step will be to define the requirements for clinical fellowships in clinical informatics and their accreditation by the Accreditation Committee for Graduate Medical Education (ACGME), which accredits residency and specialty fellowship training programs.

Papers published in JAMIA in 2009 laid out the details of the core curriculum [3] and training requirements [4] for the subspecialty. These were developed over a two-year process, funded by a grant to AMIA in 2007 from the Robert Wood Johnson Foundation. Two task forces were convened to address the core curriculum and training requirements. (I was a member of the latter.) These task forces led to the ABMS proposal that is currently under review.

Even though the process for establishing the subspecialty is well-defined, a number of questions remain. One question is how many healthcare organizations and others will require their physician-informatician practitioners to be certified. Another question, very critical to academic informatics units, is what will be the role for formal didactic education, especially that offered by distance learning. Programs such as ours at OHSU have been a popular vehicle for physicians and others to become informatics practitioners. The distance learning aspect has been especially valuable, as many clinicians enter informatics careers after they have established their clinical careers. The graduate-level education approach has been validated by the strong uptake of these programs as well as the more recent funding for them though the Office of National Coordinator for Health Information Technology (ONC) University-Based Training (UBT) Program, including the OHSU offering. I am hopeful that ACGME will adopt flexibility in the clinical informatics fellowship program educational programs, including possibly allowing organizations like OHSU to provide the coursework portion of the training requirements in settings where a large educational infrastructure is not available.

Professional recognition is important for any discipline, especially within medicine. This includes the growing number of informatics practitioners. Within medicine, the best approach to professional recognition is formal board certification. To that end, I look forward to seeing the specialty approach develop and thrive. As I am personally still board-certified in internal medicine, I hope to be able to become subcertified in clinical informatics myself.

References

[1] Detmer, D., Munger, B., et al. (2010). Clinical informatics board certification: history, current status, and predicted impact on the medical informatics workforce. Applied Clinical Informatics, 1: 11-18.
[2] Cassel, C. and Reuben, D. (2011). Specialization, subspecialization, and subsubspecialization in internal medicine. New England Journal of Medicine, 364: 1169-1173.
[3] Gardner, R., Overhage, J., et al. (2009). Core content for the subspecialty of clinical informatics. Journal of the American Medical Informatics Association, 16: 153-157.
[4] Safran, C., Shabot, M., et al. (2009). ACGME program requirements for fellowship education in the subspecialty of clinical informatics. Journal of the American Medical Informatics Association, 16: 158-166.

First Year of the OHSU University-Based Training (UBT) Program

It has been a little over a year since Oregon Health & Science University (OHSU) was awarded two grants from the Office of the National Coordinator for Health IT (ONC) Workforce Development Program. Activity on these projects has been a major part of the work in our department, and certainly of my time, over this period. In this posting, I will report on our University-Based Training (UBT) Program. In a later posting, I will report on our work on the other funded project, the Curriculum Development Centers/National Training & Dissemination Center Program.

OHSU was one of nine universities (or consortia thereof) awarded a UBT grant. We have met all of our goals and timelines so far for the project. The gist of our funded proposal was to enroll students into our Graduate Certificate and Master of Biomedical Informatics (MBI) programs, with additional course requirements based on the specific ONC workforce roles. As OHSU is on an academic quarter system, Graduate Certificate (classified by ONC as Type 1) students are expected to complete the program in an accelerated part-time status in four quarters (one year) while MBI (classified by OC as Type 2) students are expected to complete the Master's program in six quarters (one and a half years) as full-time students. We were awarded $3.08 million to fund 135 Type 1 and 13 Type 2 students over three years through 2013. The "ad" on this page links to the Web page describing the program.

We accepted 12 Type 1 students to start in the summer quarter of 2010, with 11 of those students expected to graduate in June, 2011, along with two additional students who completed the program in an accelerated manner. We will also have one Type 2 student graduating in June, 2011. These 14 graduates will be eligible (and encouraged!) to attend the OHSU June 6, 2011 Commencement.

We have an additional 74 Type 1 students in the pipeline who started the program in the fall (34), winter (26), and spring (14) quarters. These students, along with eight Master's students, are for the most part on track to graduate on time.

The students we have accepted have a great deal of geographic and occupational diversity. Similar to our distance learning program in general, our UBT students reside all across the United States. (We actually have distance learning students living in 40 states as well as six countries.) Our UBT students reside in 20 different states, with some over-representation in our region, probably reflecting proportions of applicants. Those states with more than one student in our UBT program include:

• Oregon - 39 (49%)
• Washington - 7 (9%)
• California - 4 (5%)
• New York - 4 (5%)
• Texas - 3 (4%)
• Maryland - 3 (4%)
• Tennessee - 2 (3%)
• Utah - 2 (3%)
• Virginia - 2 (3%)
• Minnesota - 2 (3%)
• Wisconsin - 2 (3%)

Our students also come from many diverse career backgrounds. While the majority come from healthcare fields, a decent-sized minority do not, and some have highly technical backgrounds who are coming to learn how information technology is applied in healthcare settings. The backgrounds with more than one representative include:

• Medicine (Physician) - 16 (20%)
• Nursing - 13 (16%)
• Business/Management - 10 (13%)
• Liberal Arts/Humanities - 6 (8%)
• Computer Science - 6 (8%)
• Public Health - 4 (5%)
• Biochemistry/Biology/Chemistry - 4 (5%)
• Finance/Accounting - 2 (3%)
• Health Information Mgmt - 2 (3%)
• Healthcare Management/Administration - 2 (3%)

Our students also have a variety of highest degrees, with over half having a graduate-level degree already. The distribution of highest degrees is as follows:

• Bachelors - 34 (44.1%)
• Masters - 24 (31.1%)
• MD - 16 (20.8%)
• PhD - 3 (3.9%)
• Other healthcare doctorate - 1 (1.3%)

One of the challenges we have faced is the competitive admissions process. We have had many more qualified applicants than we have funded positions for, so we have not been to fund some highly qualified applicants. The rate of acceptance has been 34 out of 162 (21.0%) for the fall, 26 out of 102 (25.5%) for the winter, and 14 out of 73 (19.2%) for the spring.

Moving forward, we are on track to have an additional 34 Type 1 graduates at the end of the summer quarter in early September, 2011. On September 9-10, OHSU plans to hold an informatics program reunion event, celebrating the 15 year anniversary of our first informatics degree program and the first graduates of our UBT program. Additional students will graduate later this year and into 2012, including our initial cohort of MBI students.

All told, we have committed 78 of our 135 (57.8%) Type 1 slots and eight of our 13 (61.5%) Type 2 slots. We are taking the summer quarter off for new admissions and will be admitting Type 1 and Type 2 students starting again in the fall quarter. We will award the rest of our funded slots during the 2011-2012 academic year, aiming to have everyone complete the program by the end of grant in April, 2013. During this time, our existing program is still operational, and those not awarded UBT funding can still enroll as self-funded students.

We have also implemented practicum (for Graduate Certificate students) and internship (for Master's students) programs . These programs are being administered by an Internship Coordinator whom we have hired. Students are required to find their own practicum or internship, although we help them however we can. The hosting organizations so far include health care organizations, regional extension centers, and vendors.

Another hire is our career counselor, who will help students identify and apply for jobs. We also hope this individual will collaborate with the internship coordinator as well as lay the foundation for continued relationships with employers beyond the end of the UBT funding.

All told, we are pleased with what we have accomplished in the ONC UBT program. We hope this will lead to a sustainable increased interest in biomedical informatics education and careers beyond the end of the grant itself.

Professional Science Masters: The Direction for Masters-Level Professional Degrees in Informatics?

This week, the Department of Medical Informatics & Clinical Epidemiology (DMICE) of Oregon Health & Science University (OHSU) is hosting a regional workshop focused on Professional Science Masters (PSM) degrees and programs. While attendees will come from across the Pacific Northwest, the Oregon University System (OUS) is moving forward with development of a statewide program. We are interested in exploring whether our Master of Biomedical Informatics (MBI) might fit the bill to transform into a PSM. For more information on what a PSM is, see their Web site.

PSM programs are professional science degrees with three additional attributes:

1. "Plus" courses that provide the student skills for working in industry settings, such as business and management, writing and communications, and others
2. A rigorous internship program that replaces the traditional master's thesis or capstone
3. Guidance by an external advisory committee from industry that oversee the curriculum and/or participate in the internship program

DMICE offers graduate-level programs in the field of biomedical informatics. Although we are not formally a PSM, our existing programs, especially our MBI degree, have many of the attributes required of a PSM, namely the "plus" courses, an internship program, and an external advisory committee. We changed an MBI program requirement last year that allows a structured internship to be acceptable as the program capstone.

We were actually exploring the PSM option when the large amount of funding from American Recovery & Reinvestment Act (ARRA) for investment in health information technology came along and sidetracked these efforts. Of course, our Office of the National Coordinator for Health IT (ONC) University-Based Training (UBT) grant has many conceptual overlaps with the PSM concept, with its goal of producing informatics professionals who will develop, implement, and lead electronic health record (EHR) adoption in healthcare settings.

Of course, our informatics program is focused on more than EHR adoption, even though that is the largest need. But there are plenty of other critical needs for informatics in health and biomedicine, including in genomics, clinical and translational research, public health, consumer health, and even other clinical applications, such as telemedicine. As the UBT program reaches a steady state, and with it winding down in 2013, we are now reconsidering again the transformation of the program to an official PSM. This week's workshop will help inform our next steps.

Overview of the OHSU Biomedical Informatics Program

People sometimes ask me for a big picture overview of all the programs available in the Biomedical Informatics Graduate Program in the Department of Medical Informatics & Clinical Epidemiology (DMICE) at Oregon Health & Science University (OHSU). I provide that in this posting.

Biomedical informatics is the field that uses information and related technologies to advance individual health, healthcare, public health, and biomedical research. Students enter with a variety of backgrounds and upon graduation take jobs in a diverse array of settings, including healthcare organizations, industry, research labs, and public health agencies. The OHSU program has offerings along many dimensions.

One dimension is the degree/certificate type:

• Doctor of Philosophy (PhD) in Biomedical Informatics
• Master of Science (MS) in Biomedical Informatics
• Master of Biomedical Informatics (MBI)
• Graduate Certificate (GC) in Biomedical Informatics

A second dimension is the program track:

• Clinical Informatics (CI) - focus on health care, individual health, and public health
• Bioinformatics and Computational Biology (BCB) - focus on computational aspects of genomics and molecular biology, especially their relation to human health
• Health Information Management (HIM) - focus on Registered Health Information Administrator (RHIA) certification

A third dimension is whether the program is on-campus (oc) or on-line (ol), although the two can be co-mingled, especially by local students in the Portland area. The GC program can be done completely on-line, while the MBI program done on-line requires the student to take two on-campus "short" (one week) courses.

The following table shows the degree/certificate and track dimensions, with each cell indicating whether or not the program is offered on-campus or on-line.

Track Degree	CI	BCB	HIM
PhD	oc	oc
MS	oc	oc	oc
MBI	oc/ol	oc	oc/ol
GC	oc/ol		oc/ol

Where does the 10x10 ("ten by ten") program fit into this? The 10x10 curriculum is essentially equivalent to the introductory course (BMI 510) in the CI and HIM tracks.

More information is available on our program Web site: http://www.ohsu.edu/informatics/

Information Retrieval (Search) in Health and Biomedicine Still "Springs" Eternal

One of my earliest visions of computers in medicine was the ability to type in a question and get an answer. In 1980s, while everyone in informatics was trying to build expert systems, I followed a different dream, of being able to find clinical information seamlessly. In that decade, however, I never could have imagined being able to pull up something called a Web browser, typing in words, and getting back "pages." Especially as I can do now, with something that fits in my pocket, also makes phone calls, and is connected to something I had not yet heard of in the 1980s (before I started my informatics training) called the Internet.

This fascination guided my early research interests in the area of information retrieval. I write about it now because every spring I teach my course on this topic in the OHSU graduate program, BMI 514/614. (Hence the title of this posting.) My interest in this area resulted in dozens of scientific papers and a textbook, currently in its third edition [1]. Despite the marvel I have for today's modern systems, I always have to ask myself, Why didn't I think of the idea of ranking the output (Web pages) by how many other pages pointed to them? Had I thought of that before a couple Stanford graduate students named Brin and Page, my life might be considerably different. Or at least my wealth!

I suppose one is getting up in the years when you marvel at how things are now relative to how you remember them. I certainly recall "searching" when I was in medical school in the 1980s, which involved thumbing through the giant Index Medicus books on long shelves in the library. You would "link" to the full text by walking to a different part of the library where the journals were. If your needs were really critical, you could call on a librarian for help, who would take your request to a special computer that accessed a database somewhere (which happened to be MEDLINE, from the National Library of Medicine).

I actually did my first on-line searching in the 1980s. I was able to access PaperChase, and later Elhill, through dial-up networks, though at a price. For an even heftier price, you could get access to the full text … at least "text" in monospaced font and no figures or images. The world did advance, and by 1998 you could search Pubmed for free. (Al Gore, who actually deserves more credit in this area than his critics deny him, did the first "free" search.)

Now, of course, searching is ubiquitous. You can't even not do it, since most browsers will throw you into a search engine when you type in an invalid Web address (URL) into your browser. And the world not only searches, but searches for health information. The two major periodic surveys of health information searching show that 80% of Internet users have searched for health information for themselves, their family, or their friends [2, 3].

Of course, like many areas of informatics, while use of systems is ubiquitous, not all of the problems of systems are solved. Indeed, a few years ago I wrote a short piece on this topic [4]. As wonderful as today's search systems are, we still have many areas for improvement. In that paper, I identified four areas where grand challenges remained:

• Content - getting diverse users to the right information for the right task
• Indexing - developing better metadata to get searchers to that proper content
• Linkage - allowing navigation across multiple resources, even those of different publishing entities
• Access - making access as open as possible but still being protective of intellectual property

Just as I could not fathom the World Wide Web in the 1980s, I wonder as I write this in 2011 what the world of search and on-line knowledge access will be a decade or two from now.

References

Hersh, W. (2009). Information Retrieval: A Health and Biomedical Perspective (3rd Edition). New York, NY. Springer.
Fox, S. (2011). Health topics. Washington, DC, Pew Internet & American Life Project. http://www.pewinternet.org/~/media//Files/Reports/2011/PIP_HealthTopics.pdf.
Taylor, H. (2010). "Cyberchondriacs" on the Rise? Those who go online for healthcare information continues to increase. Rochester, NY, Harris Interactive. http://www.harrisinteractive.com/vault/HI-Harris-Poll-Cyberchondriacs-2010-08-04.pdf.
4. Hersh, W. (2008). Ubiquitous but unfinished: grand challenges for information retrieval. Health Information and Libraries Journal, 25(Suppl 1): 90-93.

What is the Evidence Base for Informatics, Health IT, and Related Areas? Some Recent Analyses

The first part of 2011 has brought a number of publications, and subsequent discussion, about the "evidence base" for the efficacy of biomedical and health informatics interventions, including electronic health records. These publications and conversations come against a backdrop of a very poisoned political environment in the United States, where everything about healthcare, including informatics, has become unfortunately very politicized. In this posting, however, I will stick to the science.

The first high-profile study of the year was the on-line posting of the Archives of Internal Medicine paper by Romano and Stafford [1], which I discussed in an earlier posting. The official publication of the paper, as well as letters about it, will be published in May, 2011.

Probably the next most high-profile study was the publication of an update of a systematic review of studies of outcomes from health information technology interventions by Buntin and colleagues [2]. This was actually the second update of an original systematic review that was published in 2006 by Chaudhry and associates [3], the first update of which was published by Goldzweig and colleagues in 2009 [4].

Systematic reviews are comprehensive reviews of all research evidence on a given area or question [5]. When studies are homogeneous enough (e.g., all studies assessing the treatment of hypertension to reduce cardiovascular disease), a mathematical technique known as meta-analysis can be performed to combine results across studies to achieve larger a sample size and more statistical power. But most areas, certainly so in informatics, have research questions too heterogeneous to enable use of meta-analysis. Nonetheless, studies can be categorized to look at general questions asked, such as efficacy of decision support to reduce medical error or access to data in a more timely manner to reduce cost of care.

The three successive systematic reviews [2-4] using relatively similar methodology have summarized outcomes of studies of health information technology (HIT) over particular time periods:

• Chaudhry, 2006 – studies from 1995-2004 [3]
• Goldzweig, 2009 – studies from 2004-2007 [4]
• Buntin, 2011 – studies from 2007-2010 [2]

As with most systematic reviews, these captured a broad net of literature and reviewed it for quality of methodology and its results.

Chaudhry et al. identified 257 studies, with the most benefit shown for:

• Adherence to guideline-based care
• Enhanced surveillance and monitoring
• Decreased medical errors

An interesting caveat of the results that the authors noted was that 25% of the identified studies came from four institutions (Partners Healthcare, Veteran's Administration, Indiana University/Regenstrief Institute, and Vanderbilt University) and there were few studies of commercial systems, raising concerns about generalizability.

In their update, Goldzweig et al. found 179 new studies. They noted comparable results to the study of Chaudhry et al., but also found an increased number of studies of patient-focused applications that ran external to EHR, e.g., Web-based care management. They note a small increase in the number of studies of commercial, off-the-shelf systems, though 20% of studies still came from the four leading institutions. They also found there was still a paucity of cost-benefit analyses.

In the new systematic review, Buntin et al. identified 154 new studies with 278 individual outcome measures. While acknowledging wide divergence of study quality and methodologies, not to mention outcomes studied, they noted that 96 (62%) of studies had positive improvement in one or more aspects of care, with 142 (92%) showing positive or mixed positive-negative outcomes. They found that the studies used quantitative and qualitative approaches, with those using statistical hypothesis testing more likely to have positive outcomes. They slightly redefined “health IT leader” institutions, but noted that a large number (28) still came from these institutions, but did decreased somewhat to 18% of the studies. Somewhat reassuring  was that the “leader” studies did not differ in methods or results from the other studies.

Buntin et al. grouped the outcomes into seven categories, noting document improvement in all of them:

• Access to care
• Preventive care
• Care process
• Patient satisfaction
• Provider satisfaction
• Effectiveness of care
• Efficiency of care

Another bit of evidence from early 2011 was a review of all eHealth systematic reviews took exception to direction and quality of evidence [6]. The authors note that many studies of eHealth, including clinical applications (i.e., health IT), had poor methodology, raising concern over validity of the results. The results echo those of a systematic review I led about telemedicine studies several years ago [7]. One concern about this new review is that its methodology of being a review of reviews might magnify poor evidence. But someone needs to reconcile this review with the one of Buntin et al. [2].

It should be noted that another line of thought has been critical of the experimental approach to evaluation of health IT. Two recent commentaries note that these approaches cannot capture the whole picture of a health IT intervention, especially ones that occur in real-world implementations in complex settings, like states or even whole countries [8, 9]. I acknowledge these criticisms, though would argue back that we should not view these approaches as either-or. There is hopefully plenty of room for all types of disciplined evaluation of informatics, with clinical trials and similar experiments

References

1. Romano, M. and Stafford, R. (2011). Electronic health records and clinical decision support systems: impact on national ambulatory care quality. Archives of Internal Medicine, Epub ahead of print.
2. Buntin, M., Burke, M., et al. (2011). The benefits of health information technology: a review of the recent literature shows predominantly positive results. Health Affairs, 30: 464-471.
3. Goldzweig, C., Towfigh, A., et al. (2009). Costs and benefits of health information technology: new trends from the literature. Health Affairs, 28: w282-w293.
4. Chaudhry, B., Wang, J., et al. (2006). Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Annals of Internal Medicine, 144: 742-752.
5. Anonymous (2011). Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC, Institute of Medicine.
6. Black, A., Car, J., et al. (2011). The impact of eHealth on the quality and safety of health care: a systematic overview. PLoS Medicine, 8(1): e1000387.
7. Hersh, W., Hickam, D., et al. (2006). Diagnosis, access, and outcomes: update of a systematic review on telemedicine services. Journal of Telemedicine & Telecare, 12(Supp 2): 3-31.
8. Greenhalgh, T. and Russell, J. (2010). Why do evaluations of eHealth programs fail? An alternative set of guiding principles. PLoS Medicine, 7(11): e1000360.
9. Patrick, J. (2011). The validity of personal experiences in evaluating HIT. Applied Clinical Informatics, 1: 462-465.

Looking Back, Moving Forward

This week marks a year ago that I woke up (on the morning of Friday, April 2nd, to be precise) to find emails in my inbox telling me that Oregon Health & Science University (OHSU) had received our two awards from the Office of the National Coordinator for Health IT (ONC) Workforce Development Program. As most readers know, those programs are going well, and I am planning to provide my occasional updates of our efforts in the programs here in the coming weeks ahead. A succinct news report of the programs, for which I was interviewed, is available from the California Health Care Foundation (CHCF).

Another anniversary of sorts is for this blog, which has now been in existence a little over two years (since March 2, 2009, to be precise). I have enjoyed having this forum to share my thoughts about topics of interest and passion to me. I have tried to create thoughtful pieces that explore various issues, and not just brief streams of consciousness.

The year 2011 also is another anniversary year, which is the 15th year of informatics degree programs being offered by OHSU. In 1996, we opened the doors to our initial Master of Science degree. Of course, we have added a number of other degrees since then, such as our Master of Biomedical Informatics (non-thesis, professional master's), PhD, and Graduate Certificate. To celebrate the 15-year anniversary, as well as the first two groups of graduates from our ONC funding, we are planning to hold a celebration in September. The event will be open to the public and is scheduled to take place on September 9-10, 2011. All alumni, students, faculty, and friends of the program will be invited, with alumni being able to present about what they are currently doing, along with a number of other keynote speakers. (Save the date!)

Of course, we are not resting on our laurels, and are quite busy with our current work, the totality of which would be much longer than anyone would want to read. I am happy to announce that the 10x10 ("ten by ten") program continues going strong, with  a number of new offerings planned to start in the next few months. One of the offerings is a general AMIA offering but the rest demonstrate the partnerships that we have built for specific offerings. As with all 10x10 courses, the offerings include the basic on-line portion of the course and an in-person session often associated with a professional meeting.

They include:

• Regular AMIA offering aimed at all audiences, starting April 27, 2011 with in-person session at any AMIA national meeting in the next year. (Next meeting in Washington, DC, with in-person session on October 23, 2011)
• Offering focused on dietitians and the area of nutrition informatics, in partnership with the American Dietetic Association (ADA), starting April 13, 2011, with the in-person session at the ADA meeting in San Diego, CA on September 24, 2011
• Offering in Singapore in partnership with Gateway Consulting, starting May 2, 2011, with the in-person session in Singapore (!) on September 14, 2011
• Offering focused on emergency physicians, in partnership with the American College of Emergency Physicians (ACEP), starting June 29, 2011, with the in-person at the ACEP meeting in San Francisco, CA on October 14, 2011

Honorable Mention

Who should be the next National Coordinator for Health Information Technology (HIT), i.e., the Director of the Office of the National Coordinator for HIT (ONC)? My name appeared recently on a list of 24 individuals nominated as a potential replacement for the current National Coordinator, Dr. David Blumenthal, who is leaving the post next month to return to academia at Harvard University.

Well, the results of the voting are in, and I was flattered to finish fifth, capturing 5.6% of the 736 votes cast. The winner of the poll was Jessica Kahn, Technical Director for HIT for Medicaid at the Centers for Medicare and Medicaid Services. Following in a close second was Dr. Marc Chasin. Vice President and Chief Medical Informatics Officer at St. Luke's Health System in Boise, Idaho. I am delighted to report that Dr. Chasin is a former student of mine, having taken the 10x10 ("ten by ten") course.

I have to admit that I am ordinarily unswayed by magazine or Web polls that are completely unscientific and really just popularity contests. Still, I was flattered to be part of this. Perhaps, as they say in show business, any publicity is good publicity.

This all said, I don't think I will be leaving Oregon Health & Science University any time soon. I have waited my whole career for the situation I and the department I lead are currently in, seeing the maturation of our field and the resources now available to support education and research endeavors within it.

I also do not envy the person who actually replaces Dr. Blumenthal. Clearly the HITECH Act has, to use the terminology of Gartner Hype Cycle, hit its peak of expectations. Some of the programs are down in the trough of disillusionment, although I am confident that most if not all of them will eventually level off in the plateau of productivity. Given the current political state of Washington, DC, where scoring political points seems to have overtaken governing and producing value from government programs, the next National Coordinator is likely to spend a good deal of time in non-productive Congressional hearings. It's not that I don't think government bureaucrats, like everyone else, need to be held accountable, but it is unlikely the primary purpose of those hearings will be to report on the value to healthcare and economy that HITECH has wrought.

Being the optimist that I am, I am not dwelling too much on the current poisoned atmosphere in Washington, DC. I will certainly defend to anyone the productive investment that has been made the federal government in HIT. In our programs funded by educational grants, skills and leadership have been imparted on a new cadre of individuals, and the curricular materials we are producing will have a lasting impact on the primary goal of biomedical and health informatics, which is to improve human health, healthcare, biomedical research, and public health with information.

Natural Language Processing: A Dream That Won't Die … and Shouldn't

One of the longest-standing dreams of informatics, dating back to the early (i.e, 1960s) era of artificial intelligence, is the use of natural language processing (NLP) to extract data about patients from clinical narrative data (e.g., progress notes, discharge summaries, etc.) in the electronic health record (EHR). The notion that you can take the narrative language of clinicians and turn it into concrete facts that can be used for clinical decision support, clinical research, quality measurement, surveillance, etc. is immensely appealing.

Alas, that dream, at least in a generalizable way, is still a dream. You can count a number of my published papers over the years as a few among the many valiant efforts. Unfortunately, the variability (or some might say mangling, especially by physicians) of language, along with the hidden context and meaning "between the lines," makes NLP a very difficult task to program in a computer.

Some, however, have managed to succeed in focused ways. For example, generalizable decision support also never succeeded but it has been found that focused decision support works quite well and is used in EHRs daily. Likewise, there have a number of focused areas where NLP has provided useful data for clinical processes.

It is in this context that I am pleased to report on another contribution to the literature of clinical NLP, which is a paper that appeared in a recent issue of the Journal of the American Medical Informatics Association (JAMIA) and is lead-authored by a former student, Mary Stanfill [1]. I am a co-author. It is always a thrill to see a student publish a peer-reviewed paper, especially one that started as a term paper in one of my courses, advanced to a capstone project in a master's degree, and ultimately ended up in one of the leading journals in our field.

This paper also makes a valuable contribution of being a systematic review of all studies that report results of "automated coding and classification." The analysis shows that there have been many efforts performed using many methods in a variety of clinical domains, with a wide range of results. Of course, this gets to a gripe I have had with clinical NLP and related text mining researchers over the years, which is that evaluation studies have not advanced much beyond measuring the accuracy (e.g., recall, precision, sensitivity, specificity, etc.) of how well systems identify concepts in the text [2]. I would prefer to see the next step in systems being evaluated, such as how well NLP can impact the tasks it might be used for, such as quality measurement programs or facilitating clinical research studies. This would be akin to the "task-oriented" studies of information retrieval systems I performed years ago, which focused on how well searchers completed tasks using retrieval systems rather than just measuring how many relevant articles they retrieved [3].

The good news is that systems using NLP are starting to be deployed in operational clinical settings or clinical and translational research programs, and there is an ever-increasing amount of real data in electronic form for them to use. In addition to a growing number of individual studies, there are also large-scale projects of which NLP is a significant part. There include:

• Informatics for Integrating Biology and the Bedside (i2b2) - a long-standing project to facilitate the use of clinical data for genomic and clinical research. One of its activities includes a yearly challenge evaluation that allows research to compare systems and results on a common task. The i2b2 challenge has looked at automatic de-identification [4], identification of smoking status [5], recognition of obesity and co-morbidities [6], and extraction of medication information [7].
• Electronic Medical Records and Genomics (eMERGE) Network - a multi-center project focused on the use of data in EHRs to facilitate the study of how genetic variability contributes to health and disease [8]. One of the foci includes the use of NLP for extracting data from clinical narratives and integrating it with other data in the clinical record. One accomplishment of this research to date has been the ability to replicate four of seven known gene-disease associations [9].
• SHARP 4 - one of the four collaborative research centers being funded under the HITECH Program to facilitate meaningful use of EHRs, with a focus on secondary use of EHR data.

Another development is the launching this year of a medical records track in the Text Retrieval Conference (TREC) annual information retrieval challenge evaluation. The track will use de-identified records developed by Wendy Chapman and colleagues.

It is also impossible to discuss this topic without acknowledging the discussion around the IBM Watson question-answering system, which recently proved its mettle in a television game show Jeopardy match [10]. IBM has announced some research partnerships that will apply Watson to medical data. This is an interesting research area, but we will need to see real research results to back up the hype [11].

While there are still challenges for clinical NLP, I believe we are seeing a convergence of new methods coupled with growing needs to make use of the increasing volume of clinical data as well as our desire to facilitate re-use of that data for many purposes, such as clinical decision support, quality measurement and improvement, clinical research, and public health reporting and surveillance. While there may be generalizable approaches yet to be discovered, I suspect that evolution will be much like clinical decision support, which has been more successful when engineered to specific domain areas. But as we have also seen with clinical decision support, the ability to perform those specific tasks successfully will be highly valuable to healthcare.

References

1. Stanfill, M., Williams, M., et al. (2010). A systematic literature review of automated clinical coding and classification systems. Journal of the American Medical Informatics Association, 17: 646-651.
2. Hersh, W. (2005). Evaluation of biomedical text mining systems: lessons learned from information retrieval. Briefings in Bioinformatics, 6: 344-356.
3. Hersh, W., Crabtree, M., et al. (2002). Factors associated with success for searching MEDLINE and applying evidence to answer clinical questions. Journal of the American Medical Informatics Association, 9: 283-293.
4. Uzuner, O., Luo, Y., et al. (2007). Evaluating the state-of-the-art in automatic de-identification. Journal of the American Medical Informatics Association, 14: 550-563.
5. Uzuner, O., Goldstein, I., et al. (2008). Identifying patient smoking status from medical discharge records. Journal of the American Medical Informatics Association, 15: 14-24.
6. Uzuner, O. (2009). Recognizing obesity and comorbidities in sparse data. Journal of the American Medical Informatics Association, 16: 561-570.
7. Uzuner, O., Solti, I., et al. (2010). Extracting medication information from clinical text. Journal of the American Medical Informatics Association, 17: 514-518.
8. McCarty, C., Chisholm, R., et al. (2010). The eMERGE Network: a consortium of biorepositories linked to electronic medical records data for conducting genomic studies. BMC Genomics, 4(1): 13. http://www.biomedcentral.com/1755-8794/4/13.
9. Denny, J., Ritchie, M., et al. (2010). PheWAS: Demonstrating the feasibility of a phenome-wide scan to discover gene-disease associations. Bioinformatics, 26: 1205-1210.
10. Ferrucci, D., Brown, E., et al. (2010). Building Watson: an overview of the DeepQA Project. AI Magazine, 31(3): 59-79. http://www.aaai.org/ojs/index.php/aimagazine/article/view/2303.
11. Anonymous (2011). IBM to Collaborate with Nuance to Apply IBM’s "Watson" Analytics Technology to Healthcare. Armonk, NY, IBM. http://www-03.ibm.com/press/us/en/pressrelease/33726.wss.

PCAST Report: What's the Big Deal?

As anyone who works in informatics knows, there is unremitting stream of reports, white papers, blog entries, and other writings from various government agencies, non-profit organizations, consultants, research organizations, and others involved in health information technology (HIT). Some of these reports promote various points of view, including policy directions, while others present interesting ideas to read. (Some do neither!)

One recent report has garnered more attention than any in the last several months (perhaps since the release of the meaningful use rules). This is of course the recent report from the President's Council of Advisors on Science and Technology (PCAST) entitled, Realizing the Full Potential of Health Information Technology to Improve Healthcare for Americans: The Path Forward, which was released in December, 2010. This report, called the “PCAST report” by many, has gained high visibility, perhaps reflecting its origin from the White House. It has led the Office of National Coordinator for HIT (ONC) to ask its HIT Policy Committee to create a workgroup tasked with collecting and analyzing public comments and making recommendations relative to current and future ONC activities.

The report states its case by noting that current HIT systems do not meet their potential, mainly due to the lack of interoperability that results from proprietary data stores (mostly in proprietary systems) that blunt the free flow of data. This is hardly new. The report goes on to advocate what it views as the critical solution to the problem, which is the development of a “universal exchange language” (abbreviated by some though not in the report as “UEL”) based on the notion of data elements being reduced to their atomic core. It advocates that each of these core elements have metadata (“data about data”) tagging that includes the element and its value along with an identity of the patient, a patient-controlled privacy designation, and other provenance information about the element.

The report has certainly generated a great deal of discussion, with most of the major organizations involved in HIT having weighed in during the ONC comment period. The report certainly piqued my interest, since I have always held the view that data is the most critical aspect of everything we do with HIT. I agree with Dr. Blumenthal that data is the “lifeblood” of medicine, and my informatics nirvana consists of data freely flowing between systems and the appropriately authorized people to use it. In my dream world, one could be in one EHR and “Save as…” the data for loading into another EHR or some other application. The data would be so interoperable that any application would recognize documents (e.g., discharge summaries or progress notes), measurements (e.g., vital signs or lab values, and structure data (e.g., prescriptions).

So where does PCAST fit into all of this? In this posting, I will review the PCAST Report, summarize the commentary and criticisms, and give my own further analysis to get others thinking (as a good educator should!).

The report begins with the usual accolades for how HIT has the potential to transform healthcare. In addition to the usual improving clinician access to patient data and decision support, involving patients in their care, and enabling public health and clinical research, the report also notes that HIT can create new markets and jobs for technology as well as support healthcare reforms, including economic changes in the system. It lays out nine use cases that benefit patients, clinicians, public health, and clinical research.

However, the report notes, HIT has fallen short of its promise for four reasons. First, most current systems are proprietary applications not easily adapted into clinical workflow, with proprietary data formats not easily exchangeable. Data is not easily disaggregated or re-aggregated. Second, most healthcare organizations focus on the internal value of EHRs and have no incentive for secondary or external uses of their data for patients, other healthcare organizations, public health, or research. A third reason is that patients have concerns about the privacy and security of information of their data. Finally, the report notes that HIT has been largely focused toward administrative functions and not on improving healthcare.

The UEL will require a common infrastructure for locating and assembling data elements of a patient’s records via secure data element access services (DEAS). Data would remain local, and DEAS would be distributed, intercommunicating, and interoperable. A single appropriately authorized query would be able to locate and assemble a patient’s record across multiple DEAS.

The essential core of the report is Chapters 4-5 (pp. 39-52), which deal with the core of the technology and privacy. Chapter 4 asserts that the approach of applications as “services” does not scale up. (Though the authors seem to violate the notion of separating the application and the data.) It is argued that a better approach for healthcare data is the UEL, which is coded in (eXtensible Mark-up Language) XML and tagged with three metadata elements (in addition to the data and its value):

• Identifying information about patient – including information enabling location of the data (not necessarily a universal identifier)
• Privacy protection information – who may access the data, for what purposes, and either in an identified or de-identified state
• Provenance of data – date, time, equipment used to collect data, personnel who collected it, etc.

The UEL would aim for semantic interoperability. While adherence to specific controlled terminology sets would not be required, it would be strongly encouraged. This would allow over time for data to truly be represented in a universal way.

DEAS activities would include “crawling, indexing, security, identity, authentication, authorization, and privacy.” Queries would be issued against all DEAS on the Internet, and results could be re-constituted into a complete picture of patient. Governments, healthcare organizations, and others would operate the DEAS. In some way, this process would act like Web search engines, although they would need to have very high recall and precision to insure all the appropriate data was retrieved, while incorrect data was not. In conclusion, the chapter claims the UEL is extensible, extractable by middleware, and will lead to innovative uses and applications.

Chapter 5 lays out the privacy and security aspects of the UEL and DEAS. In essence, each and every data element would have a patient-controlled privacy attribute, allowing access to the element and its use in identified or de-identified scenarios. All data would be encrypted, which would not only protect security, but also insert a mechanism to audit access.

A number of leading HIT organizations took the opportunity of the ONC comment period to state their positions. While all applauded the raising of awareness of issues related to data and its interoperability, they also raised a number of criticisms. Many advocated that the PCAST Report serve as a broader vision rather than holding concrete solutions.

The comments about the report can be summarized as follows:

1. The constellation of current standards, as imperfect as they are, meet many of the data-related goals laid out by the report.
2. Many of the ideas of the report, while interesting and worthy of further research, are untested. Having them be the drivers of Stage 2 of meaningful use would be a substantial change in direction and put the larger HITECH Program at risk.
3. Clinical data has context, and reducing its entirety to atomic elements has the potential to lose that context. Re-constituting it may not be possible if that context is lost.
4. Much of the context in clinical data requires that records be more document-centric or at least structured in groups of elements. Disaggregating documents could lose the context they provide.
5. While everyone agrees that structured data is most desirable, some data in healthcare is too nuanced, and unstructured text is required to describe it.
6. While industry-wide standards are important, no industry with data as complex as healthcare has tried an approach like this.
7. The notion of setting privacy at the individual element is highly problematic. Allowing the patient to choose which elements can be seen or not seen by clinicians, researchers, or others could introduce many unintended consequences. It would be preferable for the patient to specify general privacy policies that are then referenced by the data elements.
8. Patients’ views of privacy may change over time, as diseases change and their own disease course changes.
9. Search engines are imperfect. The DEAS would need to operate at high levels of recall and precision that are unprecedented for Internet-based search mechanisms.
10. While the report correctly notes that current HIE efforts are struggling, it ignores that major reasons for this, which have more to do with the lack of a business model for HIE than anything about the technology.

A number of specific comments from these organizations are poignant:

• American Hospital Association (AHA) - The report should set a broader vision rather than focus on concrete solutions. Setting privacy at element level could fracture the patient record. Tagging each piece of data could be costly and inefficient. DEAS are likely to face same challenges as HIEs, with lack of a business model. ONC should change policy direction for Stage 2 of meaningful use only with great caution.
• Radiological Society of North America (RSNA)/American College of Radiology (ACR) - Echoed many of the same comments and noted we need a uniform method to manage patient identity.
• Integrating the Healthcare Enterprise (IHE) USA - Noted that the current IHE profiles cover most of the functionality required for the 9 use cases.
• Healthcare Information and Management Systems Society (HIMSS) -Privacy is contextual and changing, especially as diseases change and information about them becomes less sensitive. Encryption of the data elements provides security and an audit trail but can adversely impact workflow. The objectives of the report might not be possible without a universal patient identifier. By atomizing data, we run risk of data becoming dissociated and not being able to detect errors, so any grouping in the source data should be maintained. Metadata tagging should be virtual and not physical. Tags should be referenced and not attached, since some (e.g., privacy) might change over time. Data elements separated from documents and records potentially robs them of their context.
• HIMSS Electronic Health Record Association (EHRA) -It is better to tag data on document or record level. The privacy approach is potentially unworkable. A large-scale effort of this approach is untested.
• AMIA - Chapter 4 provides general ideas but no details nor references. There is no evidence that this approach will lead to improved care. The report was for the most part silent about other federal agencies, especially the National Library of Medicine, which has great expertise in some aspects of the proposed approach, especially related to terminology development and usage. The report underestimates the complexity of modeling the domain of medicine. It ignores past failed efforts along similar lines, such as the caBIG caDSR project. The DEAS may not be scalable or practical. ONC should not deviate from already tight timeline of Stage 2 of meaningful use. We can learn lessons from the slow adoption of HL7 Version 3, which is not suited to efficient description of task information models. There is too much focus on healthcare and not enough on health.

Other organizations that weighed included:

• HL7
• Federation of American Hospitals (FAH)
• Center for Democracy in Technology
• Project HealthDesign
• Markle Foundation
• Clinical Groupware Collaborative
• Faster Cures
• Society for Participatory Medicine
  

And of course, a number of bloggers had things to say. As always, John Halamka provided great early summaries of the report and the deliberations of the ONC HIT Policy Committee:
http://geekdoctor.blogspot.com/2010/12/spirit-of-pcast.html
http://geekdoctor.blogspot.com/2011/01/primer-on-xml-rdf-json-and-metadata.html
http://geekdoctor.blogspot.com/2011/01/example-for-pcast.html
http://geekdoctor.blogspot.com/2011/01/general-principles-of-universal.html

In a widely cited posting, Wes Rishel noted some critical points: Information flow for patient care occurs at the level of documents. Taking elements out of larger context can lose context. PCAST data elements are actually molecules, not atoms. There are plenty of molecule definitions, these should be used. Another well-known blogger, Keith Boone, added that a good deal of what the report hopes to accomplish can be done with existing standards.

What will be the impact of the PCAST report? We will find out for sure in April when the ONC releases its analysis and plans for incorporating the report’s ideas and proposals. If nothing else, the report has led to increased discussion about the importance of data interoperability, which even its critics applaud. My hope is that there is at least an acceleration toward the vision of interoperable data that most in informatics share.

Some Supplemental Information from the PCAST Report

The nine use cases were lumped into three categories based upon to whom they provided value.

Value to patients:

• Patient on warfarin wanting to know if it is safe to take an NSAID drug for an injury.
• Woman with lung mass newly discovered in a community hospital referred to a large academic medical center.

Value to clinicians:

• Internist developing primary care medical home.
• Small practice with clinicians sharing records and communicating with patients via email.
• Cardiology clinics in a part of country collaborating to improve care for patients with recent myocardial infarction.
• Family physician embedding alerts in practice to improve preventive care.

Value to research and public health:

• Physician caring for a patient enrolled in a national clinical trial.
• FDA carrying out post-marketing surveillance of adverse reactions to drugs.
• Communities or states measuring improvement toward health goals.

The final published conclusions of the report were:

1. HHS and ONC have laid a foundation for progress under meaningful use and HITECH.
2. Achievement of goals for healthcare involves accelerated progress toward robust health information exchange.
3. Effort should now focus on development of a universal exchange language that enables data to be shared and re-assembled across institutions, subject to strong privacy safeguards based on patient privacy preferences.
4. Creating these requirements is technically feasible.
5. ONC should move rapidly to develop these capabilities for stages 2 and 3 of meaningful use.
6. CMS should modernize and restructure its IT platforms to serve as a driver for progress in health IT.