Tuesday, August 26, 2014

Beyond Prediction: Data Analytics/Data Science/Big Data Must Demonstrate Value

One of my ongoing concerns for data analytics/data science/Big Data in biomedicine and health is that despite the growth of articles and other writing, the accomplishments of using these tools, especially as would be documented in peer-review journals, continues to be small. I am as enthusiastic as anyone about the prospects for harnessing the growing quantity of data in our operational electronic health record (EHR) and other systems for improving health, healthcare, and research. Yet I also believe that we need to be careful that our enthusiasm does not lead to overselling or outright hype, and that we must demonstrate the value for using data just as we would any other clinical process or tool.

There have been some news reports of the value of using Big Data. However, it would be better to see peer-review publication of such results. From the news, it has been reported that two states, Wyoming and Washington, have shown reduced emergency department visits using data-based methods, while Beth Israel Deaconess Hospital has used data as part of an effort that has helped reduce hospital readmissions by 25%. Another earlier news article reported that IBM Watson has learned from data how to diagnose cancer more accurately than physicians, although when I emailed the physician to whom that quote of its success was attributed, he replied that he had never said it (Samuel Nussbaum, email communication, July 28, 2014).

There also continues to be a spate of well-done research demonstrating the predictive value of data. Just this past week, as I was preparing this post, two interesting and informative studies of prediction came across the wire, one looking at risk for metabolic syndrome in a database of 36,944 individuals maintained by a large health insurer [1] and another looking at prediction of hospital readmission [2]. These studies are important, but all of this must be followed with implementation of approaches that make use of data to show real benefit, such as improved patient outcomes, improved health, or even cost efficiency. The best study to date I am aware of applying predictive data analytics in an effort to improve outcomes was unable to show benefit [3]. Maybe I am wrong that other studies demonstrating the application of Big Data techniques have shown benefit (or have been done at all), and I will certainly stand corrected if there are.

Despite the lack of studies demonstrating benefit, there have been plenty of interesting writings about Big Data. Some publications that have even devoted issues or volumes to the topic. One of these was the July issue of the health policy journal, Health Affairs. There were a number of interesting articles in the issue, although none reported any research results demonstrating the value of Big Data. Among the interesting papers were:
  • Bates et al. detailing what they consider the six most important use cases for Big Data: high-cost patients, readmissions, triage, patient decompensation, adverse events, and treatment optimization for diseases affecting multiple organ systems [4]
  • Krumholz describing the need for new thinking and training (including informatics) in the application of Big Data [5]
  • Curtis et al. discussing four large national multi-purpose data networks that could have substantial impact [6]
  • Longhurst et al. presented the concept of the "Green Button," a tool in the EHR that would aggregate data in an attempt to answer clinical questions for which no prior evidence existed [7]
Also appearing recently was the 2014 Yearbook of Medical Informatics, which is now available via open-access publishing and was devoted this year to the topic of Big Data. Similar to the Health Affairs issue, there were several interesting papers (including one of which I was a co-author that focused on how informatics education must adapt to Big Data [8]) but none reporting patient or organizational benefits of Big Data.

There also continues to be a steady stream of other papers related to re-use of clinical data that provide insights or demonstrate the challenges to working it. Two of these papers come from a recent special issue of Journal of the American Medical Informatics Association (JAMIA) devoted to "high-throughput phenotyping." A paper by Richesson et al. documents the challenges in something so seemingly simple as definitively determining patients diagnosed with diabetes mellitus [9]. Another paper by Pathak et al. documents the detailed work required to standardize and normalize data in the EHR for a single quality measure assessing a serum cholesterol levels below 100 mg/dL for patients with diabetes mellitus [10]. Other recent papers in JAMIA have documented the challenges with the quality of diabetes-related data used for quality indicators in primary care [11] and the significant quantity of non-conformance with the details of the Consolidated Clinical Document Architecture (C-CDA) that undermine interoperability [12].

Despite the slow progress, I am still confident that we will see scientific advances around data analytics/data science/Big Data in biomedicine and health. I agree with Cathy O'Neil, who writes that we should be "skeptics, not cynics" about Big Data [13]. In other words, we should approach data, and the results obtained from it, with informed skepticism. I reiterate what I have written in the past, that we must put data to use in ways that demonstrate benefit, apply a research mentality, and take into account the "provocations" of Dana Boyd, the most important of which is that we must not let the data define our questions of it, and instead seek data that will best answer our questions [14].


1. Steinberg, GB, Church, BW, et al. (2014). Novel predictive models for metabolic syndrome risk: a “big data” analytic approach. American Journal of Managed Care. 20: e221-e228.
2. Hebert, C, Shivade, C, et al. (2014). Diagnosis-specific readmission risk prediction using electronic health data: a retrospective cohort study. BMC Medical Informatics & Decision Making. 14: 65. http://www.biomedcentral.com/1472-6947/14/65.
3. Amarasingham, R, Patel, PC, et al. (2013). Allocating scarce resources in real-time to reduce heart failure readmissions: a prospective, controlled study. BMJ Quality & Safety. 22: 998-1005.
4. Bates, DW, Saria, S, et al. (2014). Big data in health care: using analytics to identify and manage high-risk and high-cost patients. Health Affairs. 33: 1123-1131.
5. Curtis, LH, Brown, J, et al. (2014). Four health data networks illustrate the potential for a shared national multipurpose big-data network. Health Affairs. 33: 1178-1186.
6. Krumholz, HM (2014). Big data and new knowledge in medicine: the thinking, training, and tools needed for a learning health system. Health Affairs. 33: 1163-1170.
7. Longhurst, CA, Harrington, RA, et al. (2014). A 'green button' for using aggregate patient data at the point of care. Health Affairs. 33: 1229-1235.
8. Otero, P, Hersh, W, et al. (2014). Big Data: Are Biomedical and Health Informatics Training Programs Ready? Yearbook of Medical Informatics 2014. C. Lehmann, B. Séroussi and M. Jaulent: 177-181.
9. Richesson, RL, Rusincovitch, SA, et al. (2013). A comparison of phenotype definitions for diabetes mellitus. Journal of the American Medical Informatics Association. 20: e319-e326.
10. Pathak, J, Bailey, KR, et al. (2013). Normalization and standardization of electronic health records for high-throughput phenotyping: the SHARPn consortium. Journal of the American Medical Informatics Association. 20: e341-e348.
11. Barkhuysen, P, deGrauw, W, et al. (2014). Is the quality of data in an electronic medical record sufficient for assessing the quality of primary care? Journal of the American Medical Informatics Association. 21: 692-698.
12. D'Amore, JD, Mandel, JC, et al. (2014). Are Meaningful Use Stage 2 certified EHRs ready for interoperability? Findings from the SMART C-CDA Collaborative. Journal of the American Medical Informatics Association. Epub ahead of print.
13. O'Neil, C (2013). On Being a Data Skeptic. Sebastopol, CA, O'Reilly. http://www.oreilly.com/data/free/being-a-data-skeptic.csp.
14. Boyd, D and Crawford, K (2011). Six Provocations for Big Data. Cambridge, MA, Microsoft Research. http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1926431.

Monday, August 25, 2014

Healthy Living is Not Alternative Medicine, and Vice Versa

Part of my original interest in a medical career emanated from my interest in personal health. Starting with being a distance runner in high school, developing an interest in nutrition in college, and taking charge of my middle-age weight gain a decade ago, I have always been interested in healthy living.

My early interest in health also led me to develop an interest in complementary and alternative medicine (CAM). In fact, CAM was part of what led me to a medical career, as my initial interest in computers starting in high school waned while the anti-establishment appeal of CAM attracted me as a college student in the late 1970s. Even my attraction to evidence-based medicine (EBM) comes from an adage that appealed to me from the CAM world, which was, "Let truth be your authority, not authority be your truth."

I had a resurgence of interest in the 1990s when the National Institutes of Health (NIH) ramped up its National Center for Complementary and Alternative Medicine (NCCAM) in an effort to bring some scientific rigor and objectivity to the study of CAM. I became involved in some CAM research and education activities at Oregon Health & Science University (OHSU).

Alas, I think it is fair to say that the evidence for CAM interventions still seems to be wanting. I recognize there are limits to EBM and its main tool, the randomized controlled trial (RCT), in assessing CAM therapies. But there is no reason why some CAM therapies should not show some success in RCTs. However, when put to objective evidence-based testing, most of the major CAM therapies do not hold up, including homeopathy [1], acupuncture [2], and antioxidant supplements [3]. While some may argue that taking vitamin supplements is not CAM, they too show no benefit in primary prevention of disease [4]. A number of science-based books have also reviewed the evidence base for CAM and explained research findings in lay terms [5, 6]. One of the most prolific science-based reviewers of CAM studies is Edzard Ernst MD, PhD, a physician formerly employing homeopathy whose Web site is a running commentary on CAM studies and their interpretation.

There is also increasing criticism of the research funding allocated to NCCAM. There is concern not only that NCCAM studies are not justified by the underlying science, but also that few of the studies, especially RCTs, actually have their results published [7, 8]. This has led some prominent researchers to argue that we need science-based evidence more than evidence-based medicine, i.e., RCTs are other evaluative studies that are based on sound scientific underpinnings [9]. (Biological plausibility is one of the tenets of evidence-based medicine, but seems to get lost in the desire to satisfy the clamoring for studies of CAM.)

To their credit, advocates of CAM have always been among the loudest proponents of healthy living, although I have seen my share of exceptions in CAM practitioners who eat poor diets, smoke, or otherwise live unhealthfully. I see even more of the disconnection between use of CAM and healthy living in individuals, who somehow view CAM as insurance against disease from poor health habits.

Unlike most CAM, however there is evidence to support the health benefits of true healthy living. This is very distinct from health benefits of CAM. I do not view healthy living as some form of "alternative medicine." I also recognize that not all health problems are due to poor lifestyle choices. While I am confident that my healthy lifestyle will likely contribute to my better health and longevity, I know there are plenty of medical conditions that have little to do with lifestyle.

So in contrast to CAM, there is very good evidence on a number of fronts that healthy living is associated with better health and longevity. Last year the American College of Cardiology/American Heart Association published a comprehensive guideline to lifestyle interventions for reducing risk of cardiovascular disease [10]. The underlying systematic review exhaustively identifies the evidence supporting diets emphasizing fruits, vegetables, whole grains, low-fat dairy products, lean meats, nontropical vegetable oils and nuts, and legumes, while limiting sweets, sugary beverages, and red meat [11]. The systematic review also finds evidence for limiting saturated and trans fats as well as sodium. It also finds benefit for moderate-to-vigorous intensity exercise 3-4 times per week lasting 40 minutes per session.

Although the evidence for healthy living in enhancing longevity and reducing disease is strong, it is not ironclad, as it is very difficult to perform controlled trials of healthy living. Therefore a good deal of the evidence comes from large-scale observational studies. But this evidence is solid, including an aptly titled study from Europe, Healthy Living is the Best Revenge [12], and a recent analysis that running, an activity I enjoy, reduces all-cause and cardiovascular risk mortality [13]. The latter reaffirms the U-shaped curve showing the most benefit for moderate amounts of running akin to the level I do, i.e., between 9-19 miles per week.

Another adage that resonates with me is that while a simple healthy lifestyle is beneficial, there is little evidence-based (i.e., coming from RCTs and other strong evidence) data for many foods and supplements that are dubbed "miracles." I agree with Katz, who has written a book extolling the virtues of simple healthy living via the diet and exercise regimens supported by the evidence, along with avoidance of smoking [14]. (His advice makes me think, without evidence to support it, that there are diminishing returns from more and more devotion to minutiae of good diet, and that the simple basics probably get you most of the way toward the best health returns.) In addition, like many, I got a kick out of the recent grilling of Dr. Mehmet Oz in the US Senate [15].

The scientific evidence clearly supports the benefits of healthy living, while for the most part lacking it for alternative medicine. It is important to distinguish these two, and also to remember that even the healthiest lifestyle will not prevent all disease. For these reasons, there is still a role for conventional medicine and the research underlying it. I will personally continue to live healthfully, even if I know this will not provide immunity from all illness.


1. Ernst, E (2010). Homeopathy: what does the “best” evidence tell us? Medical Journal of Australia. 192: 458-460.
2. Madsen, MV, Gøtzsche, PC, et al. (2009). Acupuncture treatment for pain: systematic review of randomised clinical trials with acupuncture, placebo acupuncture, and no acupuncture groups. British Medical Journal. 338: a3115. http://www.bmj.com/content/338/bmj.a3115.
3. Bjelakovic, G, Nikolova, D, et al. (2013). Antioxidant supplements to prevent mortality. Journal of the American Medical Association. 310: 1178-1179.
4. Fortmann, SP, Burda, BU, et al. (2013). Vitamin and mineral supplements in the primary prevention of cardiovascular disease and cancer: An updated systematic evidence review for the U.S. Preventive Services Task Force. Annals of Internal Medicine. 159: 824-834.
5. Offit, PA (2013). Do You Believe in Magic?: The Sense and Nonsense of Alternative Medicine. New York, NY, Harper.
6. Singh, S and Ernst, E (2009). Trick or Treatment?: Alternative Medicine on Trial. London, England, Corgi.
7. Atwood, KC (2013). The Ongoing Problem with the National Center for Complementary and Alternative Medicine. Skeptical Inquirer, September / October 2013. http://www.csicop.org/si/show/ongoing_problem_with_the_national_center.
8. Mielczarek, EV and Engler, BD (2014). Selling Pseudoscience: A Rent in the Fabric of American Medicine: A Study of Federal Funding Advancing Naturopathy, Acupuncture, Chiropractic, and Energy Healing as Acceptable Medical Protocols Finds Troubling Misuse of Taxpayer Dollars. Skeptical Inquirer, May/June, 2014.
9. Gorski, DH and Novella, SP (2014). Clinical trials of integrative medicine: testing whether magic works? Trends in Molecular Medicine. Epub ahead of print.
10. Eckel, RH, Jakicic, JM, et al. (2014). 2013 AHA/ACC Guideline on Lifestyle Management to Reduce Cardiovascular Risk. Journal of the American College of Cardiology. 129: S76-S99.
11. Eckel, RH, Jakicic, JM, et al. (2013). 2013 Report on Lifestyle Management to Reduce Cardiovascular Risk: Full Work Group Report Supplement. Journal of the American College of Cardiology. 129: Supplement. http://circ.ahajournals.org/content/suppl/2013/11/07/01.cir.0000437740.48606.d1.DC1/Lifestyle_Full_Work_Group_Report.docx.
12. Ford, ES, Bergmann, MM, et al. (2009). Healthy living is the best revenge: findings from the European Prospective Investigation Into Cancer and Nutrition-Potsdam study. Archives of Internal Medicine. 169: 1355-1362.
13. Lee, DC, Pate, RR, et al. (2014). Leisure-time running reduces all-cause and cardiovascular mortality risk. Journal of the American College of Cardiology. 64: 472-481.
14. Katz, DL (2013). Disease-Proof: The Remarkable Truth About What Makes Us Well. New York, NY, Hudson Street Press.
15. Haiken, M (2014). Dr. Oz's 10 Most Controversial Weight Loss Supplements. Forbes Magazine, June 18, 2014. http://www.forbes.com/sites/melaniehaiken/2014/06/18/dr-oz-senate-scolding-his-10-most-controversial-weight-loss-supplements/.

Friday, July 25, 2014

Proposing the Addition of a Standard Occupational Classification (SOC) Code for Informatics

About a decade ago, as my interests and work activity began to focus more on informatics education and workforce development, I started to ask questions about the size, scope, and required education of that workforce. Despite seeing great interest in the informatics education programs at Oregon Health & Science University (OHSU), I could find very little data about how many people were working in the field, how many more were needed, what their job activities were, or what knowledge and skills they required. I noted these problems in the first paper on this topic I published [1], and then tried to answer some of the questions on the size of the workforce with the best data I could find, which was the HIMSS Analytics Database. This led to my widely publicized finding of a need for at least 40,000 more health information technology professionals [2], which was part of the motivation for including workforce development in the Health Information Technology for Economic and Clinical Health (HITECH) Act. At the same time, I was learning that many human resources (HR) professionals were unaware of the background and skills of those working in the growing number of clinical informatics roles in healthcare organizations.

One reason for all these problems was the lack of informatics being visible in federal labor statistics. In particular, there was no Standard Occupation Classification (SOC) code for informatics. I came to learn that the importance of such codes cannot be underestimated, as they define the labor statistics maintained by the US government. They also are used by others, such as Human Resources (HR) departments in organizations to classify job offerings.

There is one code that is somewhat related to informatics, and sometimes used to point to workforce needs: 29-2071 Medical Records and Health Information Technicians. The occupations described by this code are those that "compile, process, and maintain medical records of hospital and clinic patients in a manner consistent with medical, administrative, ethical, legal, and regulatory requirements of the health care system. Process, maintain, compile, and report patient information for health requirements and standards in a manner consistent with the healthcare industry's numerical coding system." However, this code refers to the relatively low-level work of coding and maintaining medical records, and not the myriad of activities carried out by informatics professionals.

The SOC system is maintained by the US Bureau of Labor Statistics (BLS) and is revised periodically with a multi-year process. The last update was in 2010, and the informatics field was not organized enough to pursue a revision. The next update will be in 2018, and a few months ago, the government made its first call for public input for modifications to the SOC 2010 system, with recommendations by this past Monday, July 21st. For over the last year, I have been part of a team of individuals and groups (ONC, AHIMA, AMIA, and HIMSS) working to propose the inclusion of the health informatics occupation into the SOC. Our letter was submitted this week, with an AMIA press release noting the large and diverse groups supporting the inclusion.

In the process of preparing the letter, I learned a great deal about the SOC system and the process for revising it. SOC codes are supposed to describe occupations more than specific jobs. There need to be substantial numbers of people in the occupation, which must be unique from others in the SOC. The classification unfortunately has a single hierarchy, which makes it difficult to represent occupations that cross boundaries, such as health informatics. But in the end, the overwhelming sentiment of the group, one I strongly advocated as well, was that health/biomedical/clinical informatics is primarily a health professional occupation and not a computing occupation. Therefore, our overall recommendation was to add a new Health Informatics occupation residing under the major group, 29-0000 Healthcare Practitioners and Technical Occupations.

I was also pleased with several other aspects of the letter:
  • It notes that while we are asking to call the new occupation "health informatics," there are other terms, such as "biomedical informatics" and "clinical informatics," which are used to describe this occupation, and all of these all refer to the same general occupation of "health informatics."
  • There is inclusion of discussion about the new clinical informatics physician subspecialty, which not only demonstrates that informatics is important to medicine (and all health professions) but that it was not unique to any primary medical specialty.
  • It calls out the large and growing number of informatics educational programs, most of which are at the graduate level.
As noted on the BLS site, there are many more steps for revision the 2018 SOC. But it has been made clear from leaders in the field that there is an important occupation of health informatics, which is a health profession that should be included in the SOC.


1. Hersh, WR (2006). Who are the informaticians? What we know and should know. Journal of the American Medical Informatics Association. 13: 166-170.
2. Hersh, WR and Wright, A (2008). What workforce is needed to implement the health information technology agenda? An analysis from the HIMSS Analytics™ Database. AMIA Annual Symposium Proceedings, Washington, DC. American Medical Informatics Association. 303-307.

Wednesday, July 9, 2014

Competencies in Clinical Informatics for Medical Education

I wrote last year about efforts at Oregon Health & Science University (OHSU) to introduce content on clinical informatics as part of its revision of its medical school curriculum. Physician competence in clinical informatics is important for a number of reasons, such as the continuously expanding knowledge base of medicine, the need for care provided to be more accountable, and the desire of patients to interact electronically with the healthcare system the way they interact with other industries (such as retailers and banks). An additional reason for such competence for some physicians is the career opportunity provided by the designation of clinical informatics as a new medical subspecialty.

In order to integrate more clinical informatics into OHSU's curriculum, we established a working group of informatics faculty leaders to develop a set of competencies in clinical informatics. We aimed to go beyond the usual searching and basic EHR skills that increasing numbers of medical schools provide. We also wanted to focus less on mastery of the technology and more on the tasks for which it is used.

From the broad competencies, we also developed specific learning objectives and milestones, an implementation schedule, and mapping to general competency domains. After producing this material, we believed there would be value in publishing our work in a peer-reviewed journal. By doing so, we hoped that this work, and the resulting curricula, will be evaluated by ourselves and our colleagues. To this end, our published paper has just appeared (1). We chose to publish in an open-access journal so everyone can access the paper, and the publisher even provides a video abstract describing the work.

Our next steps involve implementing this new portion of our medical school curriculum. We also hope to evaluate our effort as well as learn from others who are adopt, modify, and/or evaluate our approach.

I might add that there is nothing about this work is highly specific to medical students. In other words, the competencies we have developed likely apply to all health professions students, i.e., nurses, physician assistants, pharmacists, etc.. For that matter, they also should apply to non-clinical students, e.g., health administration, public health, and so forth.


1. Hersh WR, Gorman PN, Biagioli FE, Mohan V, Gold JA, Mejicano GC. Beyond information retrieval and electronic health record use: competencies in clinical informatics for medical education. Advances in Medical Education and Practice. 2014, 5: 205-212.

Thursday, July 3, 2014

Advice to a Young Person Considering a Career in Informatics

One of the biggest challenges I face in introducing potential students to the myriad of career opportunities in biomedical and health informatics that potentially await them comes with young people. I believe that the main reason for this is this group's little exposure to our healthcare system and its myriad of problems and challenges. Like most young people, they tend to be healthy and have had very little experience with healthcare and other health-related areas. While there is little difficulty in explaining the problems that informatics tries to solve to older individuals, perhaps whose parents or children have been impacted by healthcare, or who are among the myriad of mid-career students who already work in healthcare, it is considerably more challenging to introduce someone to the importance of informatics who has had little interaction with healthcare. I was recently invited to write a chapter on the topic of introducing young people to the study of informatics, and a co-editor of the book has allowed me to reproduce it in my blog. The book will be published as: Vaidya, K., Soar, J. [eds.] 2015. Health Informatics for the Curious: Why Study Health Informatics. Canberra, Australia. Forthcoming [ISBN 978-1-925128-71-0]. What follows is an edited version of my draft chapter.

While society will always need professionals who provide hands-on care of patients, there are growing opportunities for others in health professions to contribute to not only contribute to people’s health, but also improve the delivery of healthcare and advanced research. One such profession is biomedical and health informatics, which aims to apply information and associated technologies for the benefit of health and biomedicine.

There are many trends in healthcare that demonstrate increased need for professionals trained in informatics. It begins with the person in good health who aims to maintain their health and prevent disease. If that person becomes a patient, he or she wants to interact with the healthcare system in the same way they interact with other industries such as retail or banking, i.e., through electronically connected means. For those who work in healthcare, informatics competence is needed to function in their profession, such as accessing clinical knowledge and being guided by clinical decision support. As patients, especially those with one or more chronic illnesses, are cared for by teams of individuals from home caregivers to medical subspecialists, there will be a growing need for care teams to communicate effectively and coordinate care. Likewise, as the population disperses, systems employing telemedicine and other forms of remote communication will be required.

Moving to the population level, public health authorities need to be vigilant about health-related threats, whether natural (emerging infectious diseases) or manmade (bioterrorism). And of course we will continue to require a robust medical research infrastructure, with particular promise for data-intensive research methods, such as identifying genomic causes of health and disease or leveraging the data in the growing number of electronic health record systems. As new models of healthcare financing demand more accountability for care, information systems will be required so patients can be tracked and complications can be identified and addressed early. Some combine all of the needs described in this paragraph together into the concept of the learning health system, which continuously learns based on accumulated data and its analysis [1].

A common thread across all of these trends is the growing use of data and information systems. Unlike the common uses of information technology (IT), these applications are more complex, from their need to be standardized, interoperable, and reliable as well as their requirement to protect safety and individual privacy. The field that most directly addresses these issues is what I prefer to call biomedical and health informatics [2].

A variety of data points show that professionals from this discipline are in high demand. An analysis of online job postings found 226,356 positions advertised between 2007-2011 [3]. In the meantime, a survey of healthcare CIOs shows a concern for shortages of workers in this area who have the proper skills [4]. For physicians working in this area, there is now a new medical subspecialty has been designated [5]. The nursing profession has had a specialization in nursing informatics for over a decade, and we are likely to see more certifications, as the American Medical Informatics Association (AMIA) has created a task force to develop an Advanced Interprofessional Informatics Certification that will apply to all informatics professionals, not just those who are physicians and nurses.

The occupation providing the expertise and leadership in health IT is also called, for short, informatics [2]. Other adjectives sometimes appear before “informatics” in other contexts, such as clinical informatics, biomedical informatics, bioinformatics, etc., but all generally refer to the discipline working to apply information to improve health and healthcare delivery [2]. While the occupation of informatics is fundamentally a health profession, it is not just an extension of a specific healthcare field, i.e., a physician, nurse, or allied health professional who is savvy with IT. By the same token, those who work in the occupation of health informatics are not IT professionals or managers who happen to be applying general IT skills to health or healthcare settings.

This unique occupation is increasingly valued in healthcare organizations. In the United States, for example, an analysis by the Office of the National Coordinator for Health IT of a comprehensive database of 84 million online job postings to find a total of health IT-related 434,282 job postings between 2007-2011, with 226,356 health IT core jobs and 207,926 health IT-related clinical user jobs [3]. The former would contain many who work in the occupation of informatics.

Informatics is more about information than technology, with the latter being a tool, albeit an important one, to enable better use of information. The former School of Informatics at the State University of New York Buffalo defined informatics as the Venn diagram showing the intersection of people, information, and technology. Friedman has defined a “fundamental theorem” of informatics, which states that informatics is more about using technology to help people do cognitive tasks better than about building systems to mimic or replace human expertise [6]. He has also defined informatics as “cross-training,” bridging an application domain (such as public health or medicine) with basic information sciences [7].

Within informatics are a myriad of sub-disciplines, all of which apply the same fundamental science and methods, but focused on particular subject domains. As shown in the first figure below, informatics proceeds along a continuum from the cellular and molecular (bioinformatics) to the person (medical or clinical informatics) to the population (public health informatics). Within clinical informatics may be a focus on specific healthcare disciplines, such as nursing (nursing informatics), dentistry (dental informatics), pathology (pathology informatics), etc. as well as among consumers and patients (consumer health informatics). There are also disciplines in informatics that apply across the cell-person-population spectrum:
  • Imaging informatics – informatics with a focus on imaging, including the use of PACS systems to store and retrieve images in health care settings
  • Research informatics – the use of informatics to facilitate biomedical and health research, including a focus on clinical and translational research that aims to accelerate research findings into healthcare

What are the competencies required for a career in informatics? They can be grouped into three categories, as shown in the next figure below, which broadly include health/biomedical domain knowledge, information and computing science, and people/communication skills. (This is an update of a figure I have published elsewhere, e.g., [2].)

Does one need to be a clinician to be trained and effective in a job in informatics? Must one know computer programming? The answers are no and no. Informatics is a very heterogeneous field, and there are opportunities for individuals from all types of backgrounds. One thing that is clear, however, is that the type of informatics job you assume will be somewhat dependent on your background. Those with healthcare backgrounds, particularly medicine or nursing, are likely to draw on that expertise for their informatics work in roles such as a Chief Medical or Nursing Informatics Officer. Those who do not have healthcare backgrounds still have plenty of opportunities in the field, but are more likely to end up in the wide variety of other jobs that are available.

Informatics is a career for the 21st century. There are a wide variety of jobs for people with diverse backgrounds, interests, and talents, all of whom can serve the health of society through effective use of information and associated technologies. The pathway to get to that career usually involves graduate (i.e., beyond a bachelor's degree) education, and a database of such programs is available from AMIA and includes our program at Oregon Health & Science University (OHSU).


1.     Smith M, Saunders R, Stuckhardt L, and McGinnis JM, Best Care at Lower Cost: The Path to Continuously Learning Health Care in America. 2012, Washington, DC: National Academies Press.
2.     Hersh W, A stimulus to define informatics and health information technology. BMC Medical Informatics & Decision Making, 2009. 9: 24. http://www.biomedcentral.com/1472-6947/9/24/.
3.     Schwartz A, Magoulas R, and Buntin M, Tracking labor demand with online job postings: the case of health IT workers and the HITECH Act. Industrial Relations: A Journal of Economy and Society, 2013. 52: 941–968.
4.     Anonymous, Demand Persists for Experienced Health IT Staff. 2012, College of Healthcare Information Management Executives: Ann Arbor, MI, http://www.cio-chime.org/chime/press/surveys/pdf/CHIME_Workforce _survey_report.pdf.
5.     Detmer DE and Shortliffe EH, Clinical informatics: prospects for a new medical subspecialty. Journal of the American Medical Association, 2014. 311: 2067-2068.
6.     Friedman CP, A 'fundamental theorem' of biomedical informatics. Journal of the American Medical Informatics Association, 2009. 16: 169-170.
7.     Friedman CP, What informatics is and isn't. Journal of the American Medical Informatics Association, 2012. 20: 224-226.

Wednesday, June 18, 2014

Eligibility for the Clinical Informatics Subspecialty, 2014 Update

One of the posts in this blog with the most page views ever is my January, 2013 description on eligibility for the clinical informatics subspecialty for physicians. No doubt part of the reason for its popularity was my using the post as a starting point for replying to those emailing or otherwise contacting me with questions about their own eligibility.

A year later, I still get such emails and inquiries. While the advice in the 2013 post is largely still correct, we have had the ensuing experience of the first year of the board exam, who qualified to sit for it, and what proportion of those taking the test passed. We can also put various educational offerings in context, not only for their content, but also for how the two boards qualifying physicians for the exam, the American Board of Preventive Medicine (ABPM) and the American Board of Pathology (ABP), viewed them in terms of eligibility to sit for the exam.

The official eligibility statement for the subspecialty is unchanged from last year and is documented in the same PDF file posted then from the ABPM (and summarized by the ABP). One must be a physician who has board certification in one of the primary 23 subspecialties. They must have an active and unrestricted medical license in one US state. For the first five years of the subspecialty (through 2018), the "practice pathway" or completing a "non-traditional fellowship" (i.e. one not accredited by the Accreditation Council for Graduate Medical Education, or ACGME) will allow physicians to "grandfather" the training requirements, i.e., take the exam without completing a formal fellowship accredited by the ACGME.

I have some observations about who was deemed eligible for the exam, although as always, let me give the standard disclaimer that ABPM and ABP are the ultimate arbiters of eligibility, and anyone who has questions should contact ABPM (for physicians in any specialty except pathology) and ABP (for physicians in pathology). I am only interpreting their rules.

One concern many had was the "nontraditional fellowship" for eligibility, in particular whether a master's degree in informatics would allow one to qualify. I argued that a master's degree alone should have qualified someone, since if nothing else, it (at least ours at OHSU) might meet the practice pathway time requirement, with educational time being "worth" one-half of the time of practice, and a master's degree being equivalent to at least 1-1/2 years of full-time study (i.e., 0.5 FTE over three years). (I also asserted last year that anyone education from OHSU would have the background to pass the exam. Experience bore me out, as at least 40 OHSU informatics alumni and current students - some qualifying by additional practice time in the practice pathway - passed the exam, and I am not aware of anyone from our program who did not pass it.)

We have also learned from the experience of having the first exam offered. It was exciting to see 456 diplomates newly certified in the subspecialty, including myself. However, I (and a number of others) were somewhat surprised at the pass rate of 91% for the exam being so high, given the vast body of knowledge covered by the exam and the lack of formal training, especially "book" training, of many who took the exam. It is not uncommon for pass rates for those grandfathering training requirements into a new subspecialty to be much lower. We do not know how the exam or its pass rate may change this year or beyond.

This challenges my statement in last year's posting that a single course, such as 10x10 ("ten by ten") or the American Medical Informatics Association (AMIA) Clinical Informatics Board Review Course, may not be enough. But perhaps with the experience brought to the table by qualifying via the practice pathway, a large amount of additional education is not necessary.

One bit of advice I can certainly give to any physician who meets the practice pathway qualifications (or can do so before 2018) is to sit for the exam before the end of grandfathering period. After that time, the only way to become certified in the subspecialty will be to complete a two-year, on-site, ACGME-accredited fellowship. While we are excited to be developing such a fellowship at OHSU, it will be a challenge for those who are mid-career, with jobs, family, and/or geographical roots, to up and move to become board-certified.

There are actually a number of categories of individuals for whom getting certified in the subspecialty after the grandfathering period will be a challenge:
  • Those who are mid-career - I have written in the past that the age range of OHSU online informatics students, including physicians, is spread almost evenly across all ages up to 65.
  • Those pursuing research training in informatics, such as an NLM fellowship or, in the case of some of our current students, in an MD/PhD program (and will not finish their residency until after the grandfathering period ends). Why must these individuals also need to pursue a clinical fellowship?
  • Those who already have had long medical training experiences, such as subspecialists with six or more years of training - Would such individuals want to do two additional years of informatics when, as I recently pointed out, it might be an ideal experience for them to overlay informatics and their subspecialty training?
Fortunately one option for physicians who do want some sort of certification will be the Advanced Interprofessional Informatics Certification being developed by AMIA. These physicians can and will still apply informatics to make important contributions to healthcare. I am pleased to report that AMIA has revamped its efforts to create this certification, not only for these physicians but also other practitioners of informatics.

Sunday, June 15, 2014

National Library of Medicine: Past, Present, and Future

This week I am off to a meeting I look forward to every year, the annual meeting for trainees funded by the National Library of Medicine (NLM) biomedical informatics research training grant program. This meeting has been valuable for its content and networking back to the time I first attended it as a trainee in 1988. I wrote about it in the past in this blog.

The NLM is truly an exemplar not only for the United States (which funds it via Congress), but also the rest of the world as well many people individually, including myself. The NLM is unparalleled in providing access to biomedical and health knowledge. It not only provides operational systems used around the world daily but also has a robust research and development program that develops, implements, and evaluates the next generation of biomedical and health knowledge tools. The NLM provides access not only to clinical information, but also biological, consumer-oriented, and public health data and information. As one of the institutes of the National Institutes of Health (NIH), it adds value to the research missions of all the other institutes in an integrative manner.

Starting with my postdoctoral fellowship from 1987-1990, the NLM has certainly enabled success in my career. It also funded my first research grant, one of the old First Independent Research Support and Transition (FIRST) or R29 awards, and then numerous other research as well as training grants. The NLM also funded the initial program and the building where my office resides under the Integrated Advanced Information Management Systems (IAIMS) initiative at Oregon Health & Science University. Like many in the informatics field, I owe a great deal to the NLM for my accomplishments.

At this time, the NLM is preparing for its next round of its long-term planning. This process is slated to begin in 2015 and will result in a publication of its next long-term plan in 2016. This new plan will supersede the last long-term plan that was published in 2006.

In preparation for the next round of long-term planning, the NLM recently held a symposium to reflect over the 30 years of leadership of its current director, Dr. Donald AB Lindberg. The Web site for the symposium allows visitors to leave comments about the past, present, and future of the NLM. The latter comments will be used among the information-gathering efforts to launch the next long-range planning process next year. What an opportunity for the Informatics Professor to share his thoughts on the future of the NLM, which I will put in the rest of this posting and then paste into the future comments portion of the NLM symposium site.

In light of the NLM's success, what recommendations do I suggest for moving into the future?

One of the big challenges for the NLM is its name. I am not one of those people who believes that "libraries" are something of the past. I have a whole chapter in my book on information retrieval (search) devoted to "digital libraries" and their importance [1]. Libraries are certainly changed in the 21st century, but still remain repositories of data, information, and knowledge, even if much of it is now digital. It is still important to have libraries and librarians who collect, curate, make accessible, archive, and preserve data, information, and knowledge. The fact that its materials are mostly digital now does not eliminate the need for these other functions.

However, the NLM always has been more than a library and will likely continue to be so. The NLM funds intramural (within in NLM) and extramural (outside NLM, mainly in universities) research, mostly in informatics. It also provides and funds education, mainly to librarians and future informatics researchers, but also many others. What the NLM really does then, in the big picture, is biomedical and health informatics.

Since the NLM is one of the institutes of the NIH, we may ask then, why is the NLM not called something like (my preferred name) the National Institute for Biomedical and Health Informatics? This would not only reflect its larger activities beyond being a library, but also make its role more clear, especially for those to whom NIH research funding is important (such as Promotion and Tenure Committees at universities). If this name change were made, the library function of NLM could be one of its major divisions, synergizing with its other informatics functions.

Another important recommendation for the future is to expand informatics research funding. If for no other reason, this should be done to give informatics research a fairer share of NIH research funding. Is informatics research really only deserving one-tenth or one-hundreth the funding of the major disease-based institutes? I know that substantial expansion of overall funding is unlikely to come to the NIH any time soon, but consideration should be given to re-prioritizing the role of information systems and technology in achieving the "triple aim" of healthcare (better health, better care, and lower cost) [2].

In teaching to various audiences, I have noted an analysis by Woolf and Benson comparing the overall health benefit of more people having access to a treatment versus the incremental improvement in treatment efficacy [3]. This analysis is mainly used to make the case for investments in universal healthcare (something with which I agree), but can also be applied to the notion that informatics research helps us understand how to provide better access to patient data, along with information and knowledge to deliver more effective care.

I also would advocate that the NLM (or this new institute) should expand its research explicitly in clinical informatics. I argued recently in this blog that the NIH informatics research agenda needed to expand its focus on the role that data, information, and knowledge play in complex clinical settings. I also expressed concern that data science not taking into account the context of health and healthcare might not achieve its potential.

In fact, this newly renamed institute could also be the home of other current activities, such as data science, which is so dependent on informatics and would also synergize with its library function. At a time when many organizations are developing matrixed organizations where the different entities perform integrative functions, NLM can and should take on that role within NIH, with its core function centered around informatics.


1. Hersh, WR (2009). Information Retrieval: A Health and Biomedical Perspective (3rd Edition). New York, NY, Springer.
2. Berwick, DM, Nolan, TW, et al. (2008). The triple aim: care, health, and cost. Health Affairs. 27: 759-769.
3. Woolf, SH and Johnson, RE (2005). The break-even point: when medical advances are less important than improving the fidelity with which they are delivered. Annals of Family Medicine. 3: 545-552.

Monday, June 9, 2014

What is the Right Amount of Profit in Healthcare?

I have written in the past that while free markets and capitalism work well in most industries, their value in healthcare is less clear. Other industries set prices that balance cost of production and how much consumers are willing and able to pay. I can, for example, decide when my budget allows me to buy a new car or a new computer. However, I do not believe I could ever put a price on a treatment that would save my life.

A new situation has come to the fore that reinforces this view, which is the release of the drug Sovaldi (sofosbuvir). This drug is curative of Hepatitis C in 90% of patients with the infection and has modest side effects [1]. Hepatitis C is a widespread, devastating disease that is mostly symptomatic yet can insidiously cause cirrhosis and liver failure. This drug is truly miraculous for those with this infection.

The problem with this situation is that the cost of the drug has been set by its manufacturer at $1000 per pill, meaning that the standard 12-week course costs $84,000. The drug manufacturer, Gilead Pharmaceuticals, counters that the drug saves the cost of complications and treatment of the disease, up through the use of liver transplantation that costs 3-4 fold ($300,000) the cost of the drug course along with a lifetime of expensive anti-rejection medicine ($40,000 per year) [2].

This really gets to the crux of the dilemma: What is the right amount of profit due to the innovation developed by a pharmaceutical company like Gilead? And when companies carry out less innovative activities, such as development of "me too" drugs [3], should we penalize them?

Sovaldi is not the first drug for which this dilemma has arisen. The cost of cancer chemotherapy, even when there are some competing alternatives, is extraordinarily expensive. Can the market really put a price on drugs that save or extend lives, for which there are few or no alternatives? A number of leading cancer researchers, including OHSU's Dr. Brian Druker, have raised alarms about the prices of cancer drugs [4, 5]. A Forbes Magazine contributor has discussed these issues in the context of both Sovaldi and cancer drugs [6].

This scenario has also played out with drugs for AIDS in Africa, which was documented in the movie, Fire in the Blood. Fortunately in this situation, funding from the US government came to the rescue, with the President's Emergency Plan for AIDS Relief (PEPFAR) initiative by former President George W. Bush credited with success [7]. But there are still many other challenges for high-cost drugs in developing countries.

Asking whether drug companies are greedy or innovative is probably the wrong question. If one accepts that innovation in medicine is risky and should be rewarded when it is successful, and that the cost of drug development is extraordinarily high, with a serious cost for failures (that must be spread across successes for a company's bottom line), then companies such as Gilead should indeed be rewarded. The right question is, how much should they be rewarded?

The answer gets back to the crux of medicine not adhering to the principles of a free market. When someone has a disease, especially a life-threatening but highly treatable one, he or she does not really have "choice" to choose whether or not to treat their disease? If there is just a single drug treatment, then that person is at the total mercy of the company selling the drug. The same holds for any other aspect of treatment, including the cost of physicians [8].

One possible solution to this problem is to adapt a program that has been proposed for drug development in the developing world and reward those who take the risks to develop new treatments by a measure of their health impact. One organization has proposed a plan that creates a fund to reward innovations based on their health impact globally [9, 10]. This is an intriguing idea, even if there are many challenges in the details of implementing such a model.

There are probably other solutions, but clearly society must develop a mechanism to reward true innovation and health benefits while not allowing those who have made the discovery to engage in predatory pricing. Unless solutions are developed, the current situation is only likely to exacerbate, as new discoveries in personalized [11] and precision [12] medicine emerge, which are unlikely to be developed without substantial cost.


1. Sulkowski, MS, Gardiner, DF, et al. (2014). Ledipasvir and sofosbuvir for 8 or 12 weeks for chronic HCV without cirrhosis. New England Journal of Medicine. 370: 1879-1888.
2. LaMattina, J (2014). What Price Innovation? The Sovaldi Saga. Forbes, May 29, 2014. http://www.forbes.com/sites/johnlamattina/2014/05/29/what-price-innovation-the-sovaldi-saga/
3. Gagne, JJ and Choudhry, NK (2011). How many “me-too” drugs is too many? Journal of the American Medical Association. 305: 711-712.
4. Pollack, A (2013). Doctors Denounce Cancer Drug Prices of $100,000 a Year. New York Times. April 25, 2013. http://www.nytimes.com/2013/04/26/business/cancer-physicians-attack-high-drug-costs.html
5. Experts in Chronic Myeloid Leukemia (2013). The price of drugs for chronic myeloid leukemia (CML) is a reflection of the unsustainable prices of cancer drugs: from the perspective of a large group of CML experts. Blood. 121: 4439-4442.
6. Munos, B (2014). Sovaldi Vs. Cancer Drugs: Price And Value In The Pharmaceutical Industry. Forbes, June 2, 2014. http://www.forbes.com/sites/bernardmunos/2014/06/02/sovaldi-vs-cancer-drugs-price-and-value-in-the-pharmaceutical-industry/
7. Anonymous (2009). How a Bush Administration Initiative to Combat HIV/AIDS Is Saving Lives. Washington Post. April 9, 2009. http://www.washingtonpost.com/wp-dyn/content/article/2009/04/08/AR2009040803706.html
8. Rosenthal, E (2014). Patients’ Costs Skyrocket; Specialists’ Incomes Soar. New York Times. January 18, 2014. http://www.nytimes.com/2014/01/19/health/patients-costs-skyrocket-specialists-incomes-soar.html
9. Banerjee A, Hollis A, Pogge T. The Health Impact Fund: incentives for improving access to medicines. Lancet. 2010; 375: 166-9.
10. Hollis, A and Pogge, T (2008). The Health Impact Fund: Making New Medicines Accessible for All. New Haven, CT, Incentives for Global Health. http://healthimpactfund.org/wp-content/uploads/2012/11/hif_book.pdf
11. Hamburg, MA and Collins, FS (2010). The path to personalized medicine. New England Journal of Medicine. 363: 301-304.
12. Anonymous (2011). Toward Precision Medicine: Building a Knowledge Network for Biomedical Research and a New Taxonomy of Disease. Washington, DC, National Academies Press.

Tuesday, May 27, 2014

Can the US Biomedical Research Enterprise Be Sustained? Implications for Informatics

Although I focus a good deal of my writing in this blog on the educational and career aspects of informatics, I believe that the research mission of academic informatics is equally important. Education and research are synergistic in successful academic departments, even in those where a majority of students pursue professional careers.

I have described some of the important issues in informatics research in various postings. For example, there is a need to solve the issue of data entry in order for informatics systems to provide usable data that can feed a host of functions. I have also noted that while data science is important to informatics, we cannot ignore the workflows that surround the best capture and use of data.

Unfortunately there are some challenges to the academic biomedical research enterprise in the US, not only in informatics, but in all fields. These were laid out nicely recently by Alberts et al. in a recent paper in Proceedings of the National Academy of Sciences [1]. The authors note there are "deep flaws" in US biomedical research, stemming mostly from the consequences of the system, operating under a perception of sustained growth in funding, whereas in realty federal research support has slightly declined in real dollars over the last few years and is unlikely to increase substantially in the years ahead.

The current situation has, in essence, reached a "Malthuisan" situation, due to the slowing growth outstripping resources. This leads to a number of consequences. The first of these is a hyper-competitive funding environment, where the success rate of grants being funded has fallen from 30-40% to the low teens. This results in scientists having to spend more time writing proposals, not to mention worthwhile and important science failing to get funded. It also leads scientists to be more conservative in the work they propose, sticking to tried and true, incremental science rather than bold but riskier innovation.

Another consequence of the current situation is increasing demands on scientist time. Because proposals are competitive, and increasing numbers of scientists are competing for funds, it means that more time must be spent in peer review of proposals. The hypercompetitive environment also requires that funders and others be more vigilant that funds are being spent properly, resulting in more regulations, paperwork, and so forth.

A final consequence of the current situation is an over-supply of trainees. In many biomedical research labs, graduate students do the lions share of the work. This has always been done under the assumption that this would train the next generation of scientists. The problem is that the hyper-competitive environment means it will be difficult for these students to launch successful careers as they start to compete with their mentors for a fixed or possibly shrinking amount of funding. In fact, as the article notes, only 20% of all PhD students in biomedical science in the US will get faculty positions or achieve independent research funding, as the system just cannot accommodate so many new researchers seeking funding.

The article notes that the enterprise needs to be brought into "sustainable equilibrium." They advocate new approaches for evaluating scientific proposals that are more efficient and reward the best and most innovative science, although it will always be a challenge to determine how these exalted few will be selected. The authors also call for less work of grants being done by graduate students and more by "staff scientist" types of positions by well-trained researchers who are not PIs of research. They also note that graduate programs need to prepare students for different career paths beyond being grant-funded academic faculty. One of those paths could be staff scientists while others might involve working in industry.

How does this problem play out in informatics and for the OHSU informatics program in particular? There is certainly no dearth of funding for research involving informatics, although much of it is applied and specific to projects. One downside to this is that there is little funding of core research in informatics that might lead to novel techniques. A related problem is that a good portion of  funding goes to very short-term ends, which is good for demonstrating progress to Congress and others who authorize funding but less so for advancing the field in the long run. As for informatics research locally at OHSU, we are doing relatively well but maintaining funding is always a struggle. One outgrowth that has achieved some success is looking for opportunity beyond government funding, as exemplified in our Informatics Discovery Lab that is developing partnerships with industry and others. We also have a track record of our advanced trainees (PhDs and postdocs) obtaining diverse employment for their skills.

One goal I have for the informatics field is raising awareness of this challenge, and how we might collectively work to advance the case for a more comprehensive approach to the important research that we undertake. Informatics is an interdisciplinary science but also requires attention to advancing its core scientific methods and results.


1. Alberts, B, Kirschner, MW, et al. (2014). Rescuing US biomedical research from its systemic flaws. Proceedings of the National Academy of Sciences. 111: 5773-5777.

Wednesday, May 14, 2014

Square Pegs into Round Holes - Challenges for the Clinical Fellowship Model for Clinical Informatics Subspecialty Training

Although the development of the clinical informatics subspecialty is an important accomplishment for the informatics field, I have a number of continued concerns about the optimal development of the subspecialty, especially now that the Accreditation Council for Graduate Medical Education (ACGME) program requirements for clinical informatics fellowships have been released. There are a number of aspects of training to be an informatician that just do not fit into the traditional model of clinical training, requiring those of us applying for program accreditation to fit proverbial square pegs into round holes.

Some of these concerns are also noted by Don Detmer and Ted Shortliffe in a new Viewpoint published in JAMA [1]. They raise concerns about:
  • The accreditation process that may fragment as a result of it being administered by nine different primary specialties
  • Program funding that will compete within healthcare institutions for other fellowship programs
  • National capacity for training subspecialists once the "grandfathering" period ends in 2018 and the only path to board certification is a full-time two-year fellowship
I have shared these and related concerns last year and the year before, and have also raised others. In the following sections, I will describe my major concerns, grouped under headings of the challenges with the rotations model, a more detailed description of the funding issues, and timing and flexibility issues.


Many of these are related to traditional model for clinical training versus the graduate educational model that the field has historically employed. For example, clinical training historically is based on the notion of "rotations," which are blocks of weeks that trainees spend in a given setting. This makes sense in clinical training, where one learns by being exposed to a steady stream of new and returning patients, whether on a hospital ward or in a clinic. An internal medicine trainee, for example, can learn quite a bit about intensive care medicine by taking care of the steady number of patients who are admitted, treated, and then discharged from the intensive care unit over weeks or months of time. This is the traditional "steeped tea" model of clinical training, where the assumption is that steeping the learner among patients in a given specialty will result in learning, a model that has been challenged in recent years [2]. (And for which informatics tools may help, in better tracking the cases seen.)

The rotations approach makes even less sense in clinical informatics. Trainees, for example, will derive value from spending time in the clinical informatics department, the IT organization, the compliance (privacy and security) departments, and so forth. As we hope to create a multi-instutional fellowship program, fellows in our program will also gain exposure to informatics in other settings (e.g., managed care organizations, the Veteran's Administration Hospital, safety net clinics, and even industry settings). It is not clear, at least to me, that prescribing fixed numbers of weeks in specific settings makes any sense.

A related issue with rotations is that most informatics/IT projects evolve over months, if not years. Just as primary care physician trainees need continuity experiences, so do informatics trainees. Because of the long-term nature of informatics projects, these trainees should be, in by opinion, spending significant amounts of time in them. But with all the rotations, clinical work, courses, and so forth, there will be only intermittent hours in the week for any sort of longitudinal informatics project experience.


I applaud Detmer and Shortliffe for raising concerns about funding, although the brevity of their paper does not allow detailed exploration of the complex challenges. Many institutions fund clinical fellowship programs with the understanding that the cost will at least partially be offset by the clinical work of the fellows. Clinical informatics fellows will certainly be able to make tangible contributions to the organizations that sponsor fellowships. However, they will have competing demands for their time, such as clinical work, rotations, education, teaching, and so forth, that will reduce the value they can provide to their sponsoring institutions.

An additional issue is that some, perhaps many, positions in these training programs will be funded by specific medical specialties within healthcare organizations. Those funding the programs are likely to want to see some tangible contributions to their informatics/IT efforts, just as clinical trainees contribute to the clinical mission of organizations (i.e., internists see patients in the Internal Medicine Clinic). But again, with the myriad of clinical work, rotations, education, teaching, and other demands for their time, the amount that fellows will be able to contribute to informatics work in their specialties may be limited.

Also a funding issue is that most institutions are interpreting Centers for Medicare and Medicaid Services (CMS) rules to prohibit these board-certified or board-eligible physicians, practicing in their primary specialties, to bill for patient care. Thus one source of financial value that trainees could provide to their organizations cannot be financially realized. Some have suggested that fellows do their clinical work in other settings, i.e., moonlighting, but to me, not being part of the informatics team at the same place they are providing patient care is a real lost opportunity.

A final funding issue concerns ACGME requirements for faculty, staff, and program leadership time. In these times of ever-tightening budgets for academic medical centers, the requirement for 2.0 FTE of faculty time for programs that may only have a handful of fellows is not realistic. I know this issue has historically been "fudged" by clinical programs where faculty "supervise" trainees in clinical settings while simultaneously providing patient care, but this will be challenging for clinical informatics fellowship programs.

Timing and Flexibility

I have other concerns beyond funding that I have raised in the past about timing and flexibility. The two-year, full-time commitment will block the post-2018 pathway for many physicians who have established jobs, practices, and families. Our distance learning program has more than a decade of experience of allowing physicians (and others) to transition into informatics training and careers at their own pace. One of the other subspecialty fellowship program directors at OHSU doubted that many physicians in training would want to complete a clinical subspecialty (such as oncology) and then spend another two years pursuing informatics. He did believe that there would be great interest in a joint fellowship where a trainee could get "credit" for more than one subspecialty by, for example, embedding clinical informatics training in the 18-24 months that subspecialty physicians often get for more flexible portions ("research") of their fellowship.

Clearly there are some people who should train solely in informatics, at full-time and for extended periods. These should be the researchers and educators who will work in academia, industry, and leadership roles. But clearly many "practicing" informaticians will need to maintain strong ties to their clinical specialties as well as be efficient in their training to become a subspecialist. The ability to overlay or combine informatics training with training in other specialties is appealing to my colleague program director mentioned above (not to mention myself).


In the long run, these issues will need to be resolved, especially if the new subspecialty is to thrive. Our approach at OHSU is to move forward, get our fellowship established and accredited, and then hope that ACGME and others will have the flexibility to allow clinical informatics fellowships and the larger field thrive.


1. Detmer, DE and Shortliffe, EH (2014). Clinical informatics: prospects for a new medical subspecialty. Journal of the American Medical Association. Epub ahead of print.
2. Hodges, BD (2010). A tea-steeping or i-Doc model for medical education? Academic Medicine. 85: S34-S44.

Friday, May 9, 2014

Accolades for the Informatics Professor - Spring 2014 Edition

I periodically toot my own horn in this blog, and enough interesting things have accumulated for me to do so now.

One accolade comes not from myself, but the data! My information retrieval (IR) colleague Jimmy Lin has developed a new tool, scholar-scraper, which builds a list of citation metrics of researchers in a given discipline (or from any list). One of his original lists is from the IR field, and some other colleagues, Allison McCoy and Dean Sittig, have created a list for biomedical informatics. The accolade for me is that I rank well both in biomedical informatics (15th as of today) as well as IR (18th as of today). Of course these rankings may change as the data changes as well as new scholars are added to each list.

I also have been quoted in articles on various topics, including an article on the new clinical informatics subspecialty that seems to have appeared in a number of clinical news publications:
(And probably others! To access the complete articles, free registration is required to get beyond the first page, although there is one version that seems to reliably appear without registration.)

I was also quoted in an article about the resignation of former Health and Human Services Secretary Kathleen Sebelius.

Sunday, April 13, 2014

Additional OHSU Contributions to Clinical Informatics Subspecialty Training

In addition to having our own clinical informatics fellowship, Oregon Health & Science University (OHSU) will be contributing to training in the subspecialty in other ways. One of main activities by which we will be contributing will be through providing courses in our biomedical informatics distance learning program to other programs. This is actually something we have been doing for a number of other university programs for several years, and now we are excited to do it for clinical informatics fellowship programs.

Our approach will be straightforward, as fellows in other programs will enroll as OHSU distance learning students. In discussion with colleagues directing the programs that will take part, the emerging preference for them appears to be our Graduate Certificate Program, which requires eight academic-quarter three-credit courses. Trainees will take one or two courses at a time. We also hope to enroll students from participating programs as a cohort and provide interactive opportunities for fellows in our program and those from other institutions who take our courses.

We anticipate fellows will be interested in a variety of our courses that are offered online, though have designated five courses as core to their studies, indicated by asterisks below:
BMI 510 - Introduction to Biomedical and Health Informatics*
BMI 512 - Clinical Information Systems*
BMI 513 - Electronic Health Record Laboratory
BMI 514 - Information Retrieval
BMI 515 - Ethical, Legal and Social issues in Biomedical Informatics
BMI 516 - Standards and Interoperability in Healthcare
BMI 517 - Organizational Behavior and Management*
BMI 518 - Project Management*
BMI 519 - Business of Healthcare Informatics*
BMI 520 - Consumer Health Informatics
BMI 521 - Public Health Informatics
BMI 523 - Clinical Research Informatics
BMI 537 - Healthcare Quality
BMI 544 - Databases
BMI 548 - Human-Computer Interaction
BMI 549 - Health Information Privacy and Security
BMI 560 - Design & Evaluation in Health Informatics

In order to help program directors determine the best course of study for their fellows, we have mapped all of our courses to American Board of Preventive Medicine (ABPM) core content in clinical informatics. For each course, the linked document shows whether the core content item is covered by lecture (L), article (A), reading (R), book (B), and/or exercise (E). The first five columns, highlighted in yellow, represent the content of the core courses as defined above. (We offer even more courses than this, some of which are only offered on-campus, but the entire list can be found in our course catalog.)

One question I commonly get from Program Directors concerns our tuition costs. Our tuition schedule for the 2014-2015 academic year is shown in the image below. The full cost for eight three-credit courses in the Graduate Certificate program is about $19,000. We hope to add value to that by facilitating interaction among fellows in our program and others.

I look forward to clinical informatics fellowship programs launching and OHSU playing a role in a number of them. Just as our distance learning program has led to a virtual community forming among our entire student population, I hope that a similar group will emerge among clinical informatics fellows.

Thursday, April 10, 2014

OHSU Launches Clinical Informatics Fellowship

I am pleased to announce that Oregon Health & Science University (OHSU) is formally launching its clinical informatics fellowship for physicians. We are now accepting applications for those wanting to start in July, 2014. This fellowship does not replace any of our existing fellowships or other educational programs, which include programs for physicians and non-physicians alike, but is another addition to the OHSU family of informatics educational programs.

The OHSU Clinical Informatics Fellowship will provide physicians with training in clinical informatics that will enable them to achieve board certification in the new subspecialty of clinical informatics. The program will follow the format of the guidelines recently published by the Accreditation Council for Graduate Medical Education (ACGME). The fellowship is currently applying to obtain ACGME accreditation, which will be awarded to programs starting later this year. Fellows will divide their time between informatics project work, didactic courses leading to the awarding of the Graduate Certificate in Biomedical Informatics, and clinical practice in their primary specialty. Per ACGME rules, this is a two-year fellowship that must be done full-time and completed on-site at OHSU.

The fellowship is affiliated with the OHSU Department of Medicine, with additional administrative support provided by the OHSU Department of Medical Informatics & Clinical Epidemiology (DMICE). Physicians of all medical specialties may apply. More information has been posted to the DMICE Web site, including a link to the application form.

As defined by the ACGME, clinical informatics is the subspecialty of all medical specialties that transforms health care by analyzing, designing, implementing, and evaluating information and communication systems to improve patient care, enhance access to care, advance individual and population health outcomes, and strengthen the clinician-patient relationship. Eligibility for subspecialty certification is not limited to any particular medical specialty. The new specialty was launched in 2013, with physicians already working in the field able to sit for the certification exam by meeting prior practice requirements. Starting in 2018, this "grandfathering" pathway will go away, and only those completing an ACGME-accredited fellowship will be board-eligible.

This new fellowship does not replace any existing OHSU informatics fellowship or other informatics educational program. It is a new addition to the OHSU family of informatics educational opportunities that includes a graduate program, a research fellowship funded by training grants from the NLM and other sources, and clinical fellowships offered by the Portland VA and Kaiser Permanente Northwest.

OHSU will also be providing educational content to other clinical informatics fellowship programs around the country through our online educational program. I will provide more information about this in the near future.

Wednesday, April 2, 2014

Who is Using the ONC Health IT Curriculum?

Who are the users of the Office of the National Coordinator for Health Information Technology (ONC) Curriculum? Clearly one audience is the community colleges who were funded by ONC to develop short-term training programs using the materials (82 originally, perhaps fewer now that ONC funding for the community college programs has ended).

Those of us developing the materials always knew there was a much wider audience for them, and this was borne out recently from a discussion thread on the public listserv of the AMIA Education Working Group. In early 2014, a list member posted a simple query: "Please respond to this message if you are using the educational content developed by ONC. It would be helpful to know what content you are using, the courses, and the name of the academic program."

There were a total of 15 distinct replies, and I collated them, removing all the names of those who posted as well as their institutions. The responses indicated a great diversity of users utilizing the ONC curricular materials in a variety of contexts and in different types of training opportunities:

1. We are using the HIPAA and History material in coursework for Psychiatric and Mental Health Nurse Practitioner (MSN) students.
- Assistant Professor, Nursing Informatics Specialty Coordinator, University

2. I use some materials on standards in a course for translational medicine students.
- Professor, Health Informatics, University

3. We are using some materials for a new health informatics course for allied health undergraduates at a university in Botswana.
- Professor, Epidemiology, University

4. I use the instance of the VA's VistA in an EHR lab course.
- Assistant Professor, Department of Medical Informatics, University

5. I used the material to develop a course on health culture and another on IT security for informatics students.
- Professor, Department of Medicine, University

6. Our Health Informatics Program is using the instance of Vista.
- Professor, Nursing Informatics, University

7. I am using sections for developing a Project Management course.
- Assistant Professor, University

8. We are using elements of Components 1 and 2 in an introductory course on the health care system.
- Instructor, Health Informatics Graduate Program, University

9. I am using portions for a Dental Informatics course in a Dental Hygiene Degree Completion Program.
- Instructor, College

10. I am currently developing a health informatics course for MSN Nursing Administration Students and am planning to use some of the lectures.
- Professor, University

11. I am using the Component 10, Fundamentals of Health Workflow Process Analysis and Redesign, in one-semester, 3-credit-hour, completely on-line course.
- Instructor, University

12. I am using it in several graduate courses: 1) Electronic Health Records - 4 unit course - Masters of Health Informatics, 2) Applied Health Informatics - 4 Unit Course - Masters and PhD in Nursing Science and Leadership, and 3) Applied Health Informatics - 4 unit course - Masters Degree Nurse Practitioner and Physician Assistance courses.
- Associate Adjunct Professor, Health Informatics Graduate Program, University

13. We are using the content in our Introduction to Health Informatics course developed as a first course for certificate and graduate students, incorporating video for several of the content areas across the 15 week modules.
- Associate Professor, University

14. ONC educational content has crossed the Pacific Ocean as well. I have used the following content:
A. Bachelor's Program in Information and Communication Technology (ICT) (Health IT Major)
A.1 Parts of Component 12 Units 1-12 for "Quality in Healthcare Organizations" class in the "Introduction to Health Care Systems" course
A.2 Parts of Component 7 Units 2-3 for "Hospital Services & Management" class in the "Introduction to Health Care Systems" course
A.3 Parts of Component 6 Unit 9 for "Departmental Information Systems and Management Information Systems" class in the "IT for Healthcare Services" course
B. Diploma and Master's Programs in Biomedical and Health Informatics
B.1 Parts of Component 1 Units 1a, 1b, 1c, 7a, for "Overview of Healthcare services" class in the "Fundamentals of Health Care and Medical Terminology" course
B.2 Parts of Component 1 Units 3a, 3b, 3c, 3d, 2c; Component 7 Unit 2a for "Operations in the Clinical Settings" class in the "Fundamentals of Health Care and Medical Terminology" course
B.3 Parts of Component 12 Units 1-12 for "Quality in Health Care Organizations" class in the "Fundamentals of Health Care and Medical Terminology" course
- Instructor, University, Bangkok, Thailand

15. I am teaching a course this semester on clinical decision support systems (required for our MS and PhD programs in Health Informatics), so I have borrowed from several of the components: EHR Component, Unit 2 on CDS; HIE Component, Unit 7 on CDS; HIM Component, Unit 5 on CDS
- Professor, University

One challenge for those using the curriculum is the sheer amount of material. Related to this is the lack of an outline of the entire curriculum. I recently had the opportunity to collate all of the components and units within them into a single outline, which I will include in this posting.

ONC Curriculum Topical Outline

1. Introduction to Healthcare and Public Health in the US
1. Introduction and History of Modern Healthcare in the US
2. Delivering Healthcare (Part 1)
3. Delivering Healthcare (Part 2)
4. Financing Healthcare (Part 1)
5. Financing Healthcare (Part 2)
6. Regulating Healthcare
7. Public Health (Part 1)
8. Public Health (Part 2)
9 Healthcare Reform
10. Meaningful Use

2. The Culture of Healthcare
1. An Overview of the Culture of Healthcare
2. Health Professionals – the People in Healthcare
3. Healthcare Settings – The Places Where Care is Delivered
4. Healthcare Processes and Decision Making
5. Evidence-Based Practice
6. Nursing Care Processes
7. Quality Measurement and Performance
8. Ethics & Professionalism
9. Privacy & Security
10. Sociotechnical Aspects:  Clinicians and Technology

3. Terminology in Health Care and Public Health Settings
1. Understanding Medical Words
2. Integumentary System
3. Musculoskeletal System
4. Blood, Lymphatic and Immune System
5. Cardiovascular System
6. Digestive System
7. Endocrine System
8. Ears, Nose, Throat, Eye and Vision
9. Nervous System
10. Reproductive System
11. Respiratory System
12. Urinary System
13. Public Health and Healthcare System Terminology
14. What is Health Information Management and Technology?
15. Electronic Health Records
16. Standards to Promote Health Information Exchange

4. Introduction to Information and Computer Science
1. Basic Computing Concepts, Including History
2. Internet and the World Wide Web
3. Computer Hardware
4. Computer Software
5. Computer Programming
6. Databases and SQL
7. Networks
8. Security
9. Information Systems
10. Future of Computing

5. History of Health Information Technology in the U.S.
1. Evolution of Health IT: The Early Years
2. Evolution of Health IT: The Modern Era
3. Evolution of Health IT: The HITECH Act
4. Evolution of Public Health Informatics
5. Evolution of Nursing Informatics and HIT Tools Used By Nursing
6. History of Electronic Health Records (EHRs)
7. History of Clinical Decision Support Systems
8. History of CPOE and E-Prescribing
9. History of Health Information Exchange
10. History of Privacy and Security Legislation
11. Software Certification and Regulation
12. History of Mobile Computing
13. History of Telemedicine
14. History of Quality Improvement and Patient Safety
15. Payment-Related Issues and the Role of HIT
16. History of Health IT Organizations

6. Health Management Information Systems
1. What is Health Informatics?
2. Health Information Systems Overview
3. Electronic Health Records
4. Computerized Provider Order Entry (CPOE)
5. Clinical Decision Support Systems
6. Patient Monitoring Systems
7. Medical Imaging Systems
8. Consumer Health Informatics
9. Administrative, Billing, and Financial Systems

7. Working with Health IT Systems (Lab)
1. Introduction & Overview: Components of HIT Systems
2. Under the Hood: Functions of HIT Systems
3. Understanding Information Exchange in HIT Systems
4. The Effective HIT System
5. Fundamentals of Usability in HIT Systems – What Does It Matter?
6. HIT Facilitated Error—Cause and Effect
7. Protecting Privacy, Security, and Confidentiality in HIT Systems
8. HIT System Planning, Acquisition, Installation, & Training:  Practices to Support & Pitfalls to Avoid
9. Potential Issues with Adoption and Installation of an HIT system
10. HIT and Aspects of Patient-Centered Care
11. Health IT in the Future

8. Installation and Maintenance of Health IT Systems (Lab)
1. Elements of a Typical EHR System
2. System Selection – Software and Certification
3. System Selection – Functional and Technical Requirements
4. Structured Systems Analysis and Design
5. Software Development Life Cycle
6. System Security Procedures and Standards
7. System Interfaces and Integration
8. Troubleshooting, Maintenance and Upgrades, and Interaction with Vendors, Developers, and Users
9. Creating Fault Tolerant Systems, Backups, and Decommissioning
10. Developing a Test Strategy and Test Plan
11. Pilot Testing and Full-Scale Deployment

9. Networking and Health Information Exchange
1. ISO Open Systems Interconnection (OSI)
2. Network Media and Hardware Communication Devices
3. National and International Standards Developing Organizations
4. Basic Health Data Standards
5. EHR Functional Model Standards
6. Health Data Interchange Standards
7. Supporting Standards for EHR Applications
8. Enterprise Architecture Models
9. Privacy, Confidentiality, and Security Issues and Standards
10. Health Information Exchange

10. Fundamentals of Health Workflow Process Analysis & Redesign
1. Concepts of Processes and Process Analysis
2. Process Mapping Theory and Rationale
3. Interpreting and Creating Process Diagrams
4. Acquiring Clinical Process Knowledge
5. Process Analysis
6. Process Redesign
7.  Facilitating Meetings for Implementation Decisions
8. Quality Improvement Methods
9. Leading and Facilitating Change
10. Process Change Implementation and Evaluation
11. Maintaining and Enhancing the Improvements

11. Configuring Electronic Health Records (Lab)
1. Migration to an Electronic Health Record System
2. Patient Care Clinical Workflow; Multiple Perspectives of Patient Care (VistA Demo)
3. Implementing Clinical Decision Support (VistA Demo)
4. Building Order Sets (VistA Demo)
5. Creating Data Entry Templates (VistA Demo)
6. Health Summary and Clinical Reminder Reports (VistA Demo)
7. Privacy and Security in the US
8. Meaningful Use and Implementation

12. Quality Improvement
1. Introduction to Quality Improvement and Health Information Technology
2. Principles of Quality and Safety for HIT
3. Introduction to Reliability
4. Reliability and Culture of Safety
5. Decision Support for Quality Improvement
6. Workflow Design
7. HIT Design to Support Teamwork and Communication
8. HIT and Infecting a Patient Safety Culture
9. HIT Implementation Planning for Quality and Safety
10. Measuring Quality
11. Data Quality Improvement
12. Learning from Mistakes. Error Reporting and Analysis and HIT

13. Public Health Information Technology
1. Overview & contribution to public health through Electronic Health Record use
2. Privacy, Confidentiality and Security of Public Health Information
3. Data Standards in Public Health Information Technology
4. Public health enabled electronic health records and the role of public health in health information exchange
5. Epidemiological databases and registries – Public health information tools
6. Biosurveillance, Situational awareness and disaster response
7. Public health reporting, alerts and decision support
8. The potential of public health IT for health promotion and chronic disease prevention
9. Quality Reporting
10. Encouraging adoption/use of population health functions for EHRs and Consumer functions for PHRs

14. Special Topics Course on Vendor-Specific Systems
1. Common commercial electronic health record (EHR) systems used in ambulatory and inpatient care settings
2. Certification of commercial Electronic Health Records (EHRs)
3. How do organizations select an EHR? Lessons from the front lines
4. Electronic Health Record (HER) Functionality
5. System and database architectures used in commercial EHRs
6. Vendor strategies for terminology, knowledge management, and data exchange
7. Assessing decision support capabilities of commercial EHRs
8. EHR Go-live strategies

15. Usability and Human Factors
1. People and technology, studies of technology
2. Requirements engineering
3. Cognition and Human Performance
4. Human factors and healthcare
5. Usability evaluation methods
6. Electronic health records and usability
7. Clinical decision support and usability
8. Approaches to design
9. Ubiquitous Computing
10. Designing for safety
11. Input and selection
12. Information visualization

16. Professionalism/Customer Service in the Health Environment
1. Customer Service in Healthcare IT
2. Professional Behavior in the Healthcare Environment
3. Overview of Communication Relevant to Health IT
4. Key Elements of Effective Communication
5. Regulatory Issues. HIPAA and Standard Precautions
6. Team and Small Group Communication
7. Conflict Resolution
8. Ethical and Cultural Issues Related to Communication and Customer Service
9. Personal Communications and Professionalism

17. Working in Teams
1. Health IT Teams: Examples and Characteristics
2. Forming and Developing a Team for HIT
3. Initial Tools for Teaming: Ground Rules & Action Plans for HIT Team
4. Team Strategies and Tools to Enhance Performance and Patient Safety: TeamSTEPPS
5. Leveraging Integration Techniques: Power of HIT Team Dynamics
6. Articulating Feedback and Feedforward: Tracking Success and Change
7. Leadership: All Members as Leaders – Leaderful Teams
8. Sharing Resources and Information: Tools to Optimize Performance of HIT Teams
9. Positioning for High Performance Teaming:  Challenges and Opportunities in the HIT Environment
10. Barriers to Success:  Reading Early Warning Signs of HIT Team Failure
11. Life Cycle of HIT Teams: Reforming and Repositioning Techniques

18. Planning, Management and Leadership for Health IT
1. Introduction to Leadership
2. The Management and Leadership Distinction
3. Key Concepts Associated with Leadership
4. Effective and Ineffective Leaders
5. Overview of the IT Strategic Planning Process
6. Achieving External Alignment
7. Team and Small Group Communication
8. Conflict Resolution
9. Purchasing and Contracting
10. Change Management

19. Introduction to Project Management
1. Overview of Health IT Projects
2. Project Life Cycles
3. Project Selection and Initiation
4. Project Planning Overview
5. Managing Project Scope
6. Managing Project Time, Cost, and Procurements
7. Managing Project Risk
8. Team Management and Communications
9. Project Monitoring and Control
10. Quality Management
11. Project Closure and Transition

20. Training and Instructional Design
1. Introduction to Training and Adult Learning
2. Needs Analysis
3. Creating a Lesson Plan
4. Selecting and Working with Media
5. Building & Delivering Effective PowerPoint Presentation
6. Assessments
7. Learning Management Systems
8. Web 2.0 and Social Networking Tools