Sunday, July 31, 2016

AMIA Unveils Advanced Health Informatics Certification (AHIC) for Broader Health Professions

While the clinical informatics physician subspecialty has been an excellent way to recognize the value of the informatics profession [1], there are clearly many important other professionals in the informatics field who deserve the same professional recognition for their knowledge and skills in using data and information to improve health and healthcare. A further step in that evolution took place recently with the unveiling of the Advanced Health Information Certification (AHIC) by the American Medical Informatics Association (AMIA).

More details about the process can be found in three papers published in Journal of the American Medical Informatics Association and made freely available on the AMIA Web site. These papers describe the rationale and process for developing the certification [2], the eligibility requirements for it [3], and an explanation on how it fits in the larger perspective of the field [4]. The AHIC is viewed as a specialization in informatics beyond one’s initial health professional training. The latter training must be at the master’s or professional doctorate level, such as an MD, PharmD, Master of Nursing, etc.. This pathway will also provide an alternative for physicians who are not eligible for the medical subspecialty, i.e., who do not have an active primary specialty certification. This includes those who never attained a formal specialty in their medical training as well as those who discontinued the practice of medicine and allowed their primary board certification to lapse. It also includes osteopathic physicians (DOs), although these physicians will eventually be eligible for the physician subspecialty as the merger between the Accreditation Council for Graduate Medical Education (ACGME) and the American Osteopathic Association (AOA) is implemented and AOA programs achieve ACGME accreditation.

To be eligible for AHIC, an individual with a health professional master’s or higher must also have a master’s degree or higher from an accredited informatics program and professional experience applying informatics to healthcare. The accreditation of informatics educational programs will be based on AMIA’s recently becoming a member of the Commission on Accreditation for Health Informatics and Information Management (CAHIIM), which is in the process of revising its health informatics accreditation standards. Similar to the physician subspecialty, a “grandfathering” period will allow individuals to achieve certification from educational programs deemed “acceptable” that are not yet accredited. There will also be a temporary pathway for those with no formal informatics education at all who are long-time practitioners, i.e., have 36 months of informatics experience over a five-year period that has been completed within the past 10 years. Those who obtain a PhD in an informatics-related field will also be eligible, even if they do not have formal health professional training.

One way AMIA hopes to see the process viewed is as analogous to the physician certification. In particular, those certified by AHIC are expected to be have advanced training in a healthcare profession in addition to formal training and experience in informatics.

AMIA will also establish an entity to develop the certification exam, which will likely be aligned with the physician subspecialty certifying exam. The content for the certification exam will be based on an update of the core content for the physician subspecialty, which itself needs update since it has not been revised since it was published in 2009 [5]. (I have always believed that there is little in the core content of the physician subspecialty that is truly specific to physicians. There really need not be any, since informatics is agnostic and complementary to one's healthcare field.)

As with the physician subspecialty, I am highly supportive of the new certification and its professional recognition of all who work in the field professionally. Our Biomedical Informatics Graduate Program at Oregon Health & Science University (OHSU) will certainly aim to align with it. (We are currently accredited under the original CAHIIM health informatics process but will transition to the new one when we are able to do so.)

Despite my optimism and support for AHIC, I do have one concern, which is the requirement to have a master’s degree in a health profession. I understand the rationale for aligning the AHIC with the physician subspecialty in requiring advanced training in both informatics and a health profession. At some level, this makes sense, and I have long advocated that informatics is primarily a health profession. However, this leaves out those with master’s degrees in applied informatics who do not also have a master’s degree in a health profession. That excludes those with pre-informatics training in non-health professional fields, such as computer science, life sciences, and health administration. It also leaves out health professionals whose health field has a bachelor’s degree as a terminal degree. I am not aware of any evidence that shows those with a healthcare master’s are any better operational informaticians than those without such a degree, nor do I know that potential employers share the view that they are different. Interestingly, those who obtain a PhD in informatics are not subject to this requirement, even though their degree is a research degree, i.e., less applied and less likely to have courses about the healthcare system.

There is no question that informaticians working in healthcare settings need to have a solid understanding of the healthcare system. Indeed, most applied informatics master’s programs (including ours at OHSU) have courses about the healthcare system, and are encouraged to obtain practical experiences in healthcare settings. I worry that this process may cleave professional informatics master’s programs in two, with those who are eligible for AHIC and those who are not, despite having nearly the exact same training. Many current and former OHSU master’s graduates without formal clinical backgrounds have developed successful careers in applied health or clinical informatics. The AMIA leadership has vowed to consider other certifications for these types of individuals.

Nonetheless, the AHIC will be a great accomplishment for the field when it is fully implemented over the next year or two. The recognition brought on by certification of individuals will advance the profession as a whole and consolidate the important contributions that informaticians bring to 21st century healthcare and other health-related activities.

References

1. Detmer, DE and Shortliffe, EH (2014). Clinical informatics: prospects for a new medical subspecialty. Journal of the American Medical Association. 311: 2067-2068.
2. Gadd, CS, Williamson, JJ, et al. (2016). Creating advanced health informatics certification. Journal of the American Medical Informatics Association. 23: 848-850.
3. Gadd, CS, Williamson, JJ, et al. (2016). Eligibility requirements for advanced health informatics certification. Journal of the American Medical Informatics Association. 23: 851-854.
4. Fridsma, DB (2016). The scope of health informatics and the Advanced Health Informatics Certification. Journal of the American Medical Informatics Association. 23: 855-856.
5. Gardner, RM, Overhage, JM, et al. (2009). Core content for the subspecialty of clinical informatics. Journal of the American Medical Informatics Association. 16: 153-157.

Sunday, June 26, 2016

20 Years of Biomedical Informatics Graduate Education at OHSU

Earlier this month was graduation at Oregon Health & Science University (OHSU), and I was proud to see 41 individuals listed in the program receiving Graduate Certificates as well as master’s and PhD degrees in biomedical informatics. This year also marks the 20th year of the program, dating back to the first group of master’s degree students matriculating in the fall of 1996. (OHSU already had been funded as part of the National Library of Medicine [NLM] training grant program since 1992, but initially only accepted non-degree postdoc trainees. I also launched my introductory course prior to 1996 as an elective in the OHSU Master of Public Health program.)

The program has achieved many milestones, but clearly the most important is the number of people who have launched careers in the field by graduating from the program. As shown in the table below, the OHSU Biomedical Informatics Graduate Program now has awarded 716 degrees and certificates to 653 people. (Some of the latter have more than one of the former.)


Later this year will be the celebration of another milestone for informatics at OHSU, which will be the celebration of 25 years of the Biomedical Information Communication Center (BICC) Building. I have worked in this building since the day it opened and look forward to celebrating the sustained success of its occupants, especially the Department of Medical Informatics & Clinical Epidemiology and the OHSU Library.

Monday, June 6, 2016

Generalizability and Reproducibility of Scientific Literature and the Limits to Machine Learning

A couple years ago, some colleagues and I wrote a paper raising a number of caveats about the enthusiasm for leveraging the growing volume of patient data in electronic health records and other clinical information systems for so-called re-use of secondary use, such as clinical research, quality improvement, and public health [1]. While we shared that enthusiasm for that type of use, we also recognized some major challenges for trying to extract knowledge from these sources of data, and advocated a disciplined approach [2].

Now that the world’s knowledge is increasingly available in electronic form in online scientific papers and other resources, a growing number of researchers, companies, and others are calling for the same type of approach that will allow computers to process the world’s scientific literature to answer questions, give advice, and perform other tasks. Extracting knowledge from scientific literature may be easier than from medical records. After all, scientific literature is written in a way to report findings and conclusions in a relatively unambiguous manner. In addition, scientific writing is usually subject to copy-editing that decreases the likelihood of grammatical or spelling errors, both of which often make processing medical records more difficult.

There is no question that machine processing of literature can help answer many questions we have [3]. Google does an excellent job of answering questions I have about the time in various geographic locations, the status of current airplane flights, and calories or fat in a given food. But for more complex questions, such as the best treatment for a complex patient, we still have a ways to go.

Perhaps the system with the most hype around this sort of functionality is IBM’s Watson from. Recently, one of the early leaders of artificial intelligence research, Dr. Roger Schank, took IBM to task for its excessive claims (really marketing hype) around Watson [4]. Among the concerns Schank raised were IBM claims that Watson can “out-think” cancer. I too have written about Watson, in a posting to this blog now four years ago, in which I lamented the lack of published research describing its benefits (as opposed to hype pieces extolling its “graduation” of medical school) [5]. While there have been some conference abstracts presented on Watson’s work, we have yet to see any major contributions toward improving the diagnosis or treatment of cancer [6]. Like Schank, I find Watson’s technology interesting, but claims of its value in helping clinicians to treat cancer or other diseases need scientific verification as much as the underlying treatments being used.

We also, however, need to do some thought experiments as to how likely computers can carry out machine learning in this manner. In fact, there are many reasons why the published scientific literature must be approached with care. It has become clear in recent years that what is reported in the scientific literature may not reflect the totality of knowledge, but instead representing the “winner’s curse” of results that have been positive and thus more likely to be published [7,8]. In reality, however, “publication bias” pervades all of science [9].

In addition, further problems plague the scientific literature. It has been discovered in recent years that a good deal of scientific experiments are not reproducible. This was found to be quite prevalent in preclinical studies analyzed by pharmaceutical companies looking for promising drugs that might be candidates for commercial development [10]. It has also been demonstrated in psychology [11]. In a recent survey of scientists, over half agreed with the statement there is a “reproducibility crisis” in science, with 50-80% (depending on the field) unable to reduce an experiment yet very few trying or able to publish about it [12].

Even on the clinical side we know there are many problems with randomized controlled trials (RCTs). Some recent analyses have documented that RCTs do not always reflect the larger population from which the sampling is intended to represent [13,14], something we can now document with that growing quantity of EHR data [15].  Additional recent work has questioned the use of surrogate outcomes in cancer drugs, questioning their validity as indicators of efficacy of the drugs [16,17]. Indeed, it has been shown in many areas of medicine that initial studies are overturned with later, usually larger, studies [18-20].

In addition, the problem is not limited to published literature. A recent study was published that documented significant amounts of inaccuracy in drug compendia that are commonly used by clinicians [21].

My argument has always been that informatics interventions must prove their scientific mettle no differently than other interventions that claim to improve clinical practice, patient health, or other tasks for which we develop them. A few years back some colleagues and I raised some caveats about clinical data. For different reasons, there are also challenges with scientific literature as well. Thus we should be wary of system that claim to “ingest” scientific literature and perform machine learning from it. While it is important to continue this important area of research, we must resist efforts to over-hype it and also must carry out research to validate its success.

References

1. Hersh, WR, Weiner, MG, et al. (2013). Caveats for the use of operational electronic health record data in comparative effectiveness research. Medical Care. 51(Suppl 3): S30-S37.
2. Hersh, WR, Cimino, JJ, et al. (2013). Recommendations for the use of operational electronic health record data in comparative effectiveness research. eGEMs (Generating Evidence & Methods to improve patient outcomes). 1: 14. http://repository.academyhealth.org/egems/vol1/iss1/14/.
3. Wright, A (2016). Reimagining search. Communications of the ACM. 59(6): 17-19.
4. Schank, R (2016). The fraudulent claims made by IBM about Watson and AI. They are not doing "cognitive computing" no matter how many times they say they are. Roger Schank. http://www.rogerschank.com/fraudulent-claims-made-by-IBM-about-Watson-and-AI.
5. Hersh, W (2013). What is a Thinking Informatician to Think of IBM's Watson? Informatics Professor. http://informaticsprofessor.blogspot.com/2013/06/what-is-thinking-informatician-to-think.html.
6. Kim, C (2015). How much has IBM’s Watson improved? Abstracts at 2015 ASCO. Health + Digital. http://healthplusdigital.chiweon.com/?p=83.
7. Ioannidis, JP (2005). Why most published research findings are false. PLoS Medicine. 2(8): e124. http://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124.
8. Young, NS, Ioannidis, JP, et al. (2008). Why current publication practices may distort science. PLoS Medicine. 5(10): e201. http://www.plosmedicine.org/article/info%3Adoi%2F10.1371%2Fjournal.pmed.0050201.
9. Dwan, K, Gamble, C, et al. (2013). Systematic review of the empirical evidence of study publication bias and outcome reporting bias - an updated review. PLoS ONE. 8(7): e66844. http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0066844.
10. Begley, CG and Ellis, LM (2012). Raise standards for preclinical cancer research. Nature. 483: 531-533.
11. Anonymous (2015). Estimating the reproducibility of psychological science. Science. 349: aac4716. http://science.sciencemag.org/content/349/6251/aac4716.
12. Baker, M (2016). 1,500 scientists lift the lid on reproducibility. Nature. 533: 452-454.
13. Prieto-Centurion, V, Rolle, AJ, et al. (2014). Multicenter study comparing case definitions used to identify patients with chronic obstructive pulmonary disease. American Journal of Respiratory and Critical Care Medicine. 190: 989-995.
14. Geifman, N and Butte, AJ (2016). Do cancer clinical trial populations truly represent cancer patients? A comparison of open clinical trials to the Cancer Genome Atlas. Pacific Symposium on Biocomputing, 309-320. http://www.worldscientific.com/doi/10.1142/9789814749411_0029.
15. Weng, C, Li, Y, et al. (2014). A distribution-based method for assessing the differences between clinical trial target populations and patient populations in electronic health records. Applied Clinical Informatics. 5: 463-479.
16. Prasad, V, Kim, C, et al. (2015). The strength of association between surrogate end points and survival in oncology: a systematic review of trial-level meta-analyses. JAMA Internal Medicine. 175: 1389-1398.
17. Kim, C and Prasad, V (2015). Strength of validation for surrogate end points used in the US Food and Drug Administration's approval of oncology drugs. Mayo Clinic Proceedings. Epub ahead of print.
18. Ioannidis, JP (2005). Contradicted and initially stronger effects in highly cited clinical research. Journal of the American Medical Association. 294: 218-228.
19. Prasad, V, Vandross, A, et al. (2013). A decade of reversal: an analysis of 146 contradicted medical practices. Mayo Clinic Proceedings. 88: 790-798.
20. Prasad, VK and Cifu, AS (2015). Ending Medical Reversal: Improving Outcomes, Saving Lives. Baltimore, MD, Johns Hopkins University Press.
21. Randhawa, AS, Babalola, O, et al. (2016). A collaborative assessment among 11 pharmaceutical companies of misinformation in commonly used online drug information compendia. Annals of Pharmacotherapy. 50: 352-359.
22. Malin, JL (2013). Envisioning Watson as a rapid-learning system for oncology. Journal of Oncology Practice. 9: 155-157.

Thursday, April 28, 2016

Earthquake Preparedness in the Pacific Northwest, 21st Century Style: Don't Forget the Data

Most of us who live in the Pacific Northwest have known for a couple decades of the earthquake risk sitting 90 miles off the Pacific coast, the Cascadia subduction zone. Concern reached a fervent pitch last year with the publication of an article in The New Yorker magazine authored by Kathryn Schulz, which was recently awarded a Pulitzer Prize. A follow-on article to the original piece provided good advice concerning planning.

At my family's household, we have always had earthquake insurance, although had never really taken planning seriously. As with many in Portland, the New Yorker article sprung us into action. A first step was to educate ourselves, i.e., what might happen and how can we best prepare. We also wanted to take into account our 21st century lifestyles that include work-related travel as well as a good portion of our lives being digitized.

There are many resources that are available to get one starting thinking and acting on the planning process. The first step is to understand the risk and what may happen. The State of Oregon Department of Geology and Mineral Industries, the Cascadia Region Earthquake Workgroup, and others have laid out the risk and likely consequences. While the most devastation will occur along the Oregon coast, the damage in Portland, 50 miles inland from the coast, will still be substantial. The figure below, from the Oregon Resilience Plan, shows the likely effects based on location.


One can quickly find a great deal of information. The State of Oregon also maintains a Web site about earthquakes. The state has also carried out some detailed analyses, The Earthquake Risk Study for Oregon's Critical Energy Infrastructure Hub, and The Oregon Resilience Plan – Cascadia: Oregon’s Greatest Natural Threat. The Cascadia Region Earthquake Workgroup has also published a report, Cascadia Subduction Zone Earthquakes: A magnitude 9.0 earthquake scenario.

So how does our family begin planning? I have found the most useful publication to be on the State Web site, Living on Shaky Ground: How to Survive Earthquakes and Tsunamis in Oregon. This report details the planning process, which our family has started to implement. Another report, Cascadia Subduction Zone Catastrophic Operations Plan, gives us a sense of planning for the region.

Our first step has been to start stockpiling water, food, and other supplies. Of course, it is impossible know how much of what we will need, what the public emergency response will be, and where we will even be when an earthquake happens. But we have started stockpiling water, canned food, and medical supplies. Our supply of materials are in plastic bins on the side of our house, some of which can be seen in the picture below. We will be adding more over time.


Another concern is how we will communicate and know we are all safe. It is likely that all telecommunications - cell phones, land lines, and Internet - will initially be disabled. Our plan is to migrate toward Oregon Health & Science University (OHSU), which is not only where three of the four of us work and attend school, but it likely itself to be an epicenter of recovery activity.

Another important action is to retrofit our house to improve its chances of surviving an earthquake and being able to escape once it starts. Many homes in the Pacific Northwest consist of a concrete slab foundation with a wood-based frame sitting atop it. Most experts recommend a retrofit process that bolts the frame to the foundation. Ironically, we “grandfathered” in on earthquake insurance back in the 1990s and never had to do this to obtain insurance. Many people now have their houses bolted in order to obtain insurance, but we are doing things backward in a sense, bolting the house even though we already have insurance.

There is uncertainty in the value of this procedure, since we will never know up until the onset of an earthquake whether it was a good investment. But it does bring some peace of mind, and even if the house does not remain livable after an earthquake, the added reinforcement will provide a greater chance of being able to escape once an earthquake starts. We recently had this process completed, with about two dozen plates installed that bolt the wood frame of the house to the concrete foundation or walls emanating from it. We also had an emergency shut-off switch installed for our gas line and bolted down our hot-water heater.

Below are some pictures of the bolting process. The first shows the east side of our house, where the upper level sites atop the garage. The garage walls are concrete, and this picture shows the plates above the garage level and before the siding was reinstalled.


The next picture shows the plates on the rear of the house, after the siding was reinstalled. There are no plates over the windows, since this part of the wall lacks strength.


On the west side of our house is a deck, which needed to be partially removed to install plates in that location.


In the area of the garage doors, the plates needed to be installed on the inside.


My work life also impacts my planning. One concern is my travel schedule, which finds me on the road once or twice per month. It is entirely possible an earthquake could happen while I was away from Oregon, which would complicate getting back after it happens. Unfortunately there is little I can do in advance for this possibility.

A final critical activity in this day and age is to preserve my data. I have always been meticulous about backing up my data, especially work data and my large personal photo and video archive. But most of the backup has been local, in particular to external hard disks at home and in my office. These may survive an earthquake, but an added layer of safety is provided by the technology of our time, namely in the cloud.

As we use Box at OHSU for cloud-based storage, I have uploaded all of my work-related data to my account there. This includes archives of data, documents, and teaching materials. My work life will obviously be interrupted in a major way by an earthquake, but preserving my data will at least give me a chance to get restarted at some point.

I also want to preserve what I can of my personal life in the form of photos, videos, and other data. These days I rarely print pictures, preferring to view them on my computer, tablet, or phone. I have purchased a 1-terabyte Dropbox account to handle archiving of all of my personal data.

Clearly a major earthquake will be devastating to my personal and professional life. But by being prepared, I am improving my chances of survival as well as returning to a somewhat normal life after it happens.

Monday, March 28, 2016

Eligibility for the Clinical Informatics Subspecialty: 2016 Update

Some of the most highly viewed posts in this blog have been those on eligibility for the clinical informatics subspecialty for physicians, the first in January, 2013 and an update in June, 2014. I recently led a Webinar hosted by the American Medical Informatics Association (AMIA) on the eligibility that I am using as a segue here to provide another update, which may be my last one before the "grandfathering" period ends.

One of the reasons for these posts has been to use them as a starting point for replying to those who email or otherwise contact me with questions about their own eligibility. After all these years, I still get such emails and inquiries. While the advice in the previous posts is largely still correct, we have had the ensuing experience of three years of the board exam, who qualified to sit for it, and what proportion of those taking the test passed. There are still (only) two boards that qualify physicians for the exam, the American Board of Preventive Medicine (ABPM) and the American Board of Pathology (ABP). ABP handles qualifications for those with Pathology as a primary specialty and ABPM handles those from all other primary specialties.

The official eligibility statement for the subspecialty is unchanged from the beginning of the grandfathering period and is documented in the same PDF file posted then from the ABPM (and summarized by the ABP). One must be a physician who has board certification in one of the primary 23 medical specialties. They must have an active and unrestricted medical license in one US state. For the first five years of the subspecialty (through 2017), the "practice pathway" or completing a "non-traditional fellowship" (i.e. one not accredited by the Accreditation Council for Graduate Medical Education, or ACGME) will allow physicians to "grandfather" the training requirements, i.e., take the exam without completing a formal fellowship accredited by the ACGME. But starting in 2018, the only pathway to board eligibility will be via an ACGME-accredited fellowship.

In the Webinar, I made some observations about who was deemed eligible for the exam, although as always, I must provide the disclaimer that ABPM and ABP are the ultimate arbiters of eligibility, and anyone who has questions should contact ABPM or ABP. I only interpret their rules.

We have now learned from the experience of having the exam offered over three years. As I noted in the Webinar, there are now 1107 physicians who have achieved certification in the subspecialty. The exam pass rate has been high but has declined each year, starting at 91% in 2013 and falling to 90% in 2014 and 80% in 2015. While we cannot be sure that the exam has not changed, the declining pass rate most likely reflects highly experienced individuals taking the exam initially and those with less experience taking it in subsequent years.

I (and a number of others) have been somewhat surprised at the high pass rate in all years, given the vast body of knowledge covered by the exam and the lack of formal training, especially "book" training, of many who took the exam. It is not uncommon for pass rates for those grandfathering training requirements into a new subspecialty to be much lower. We will have to see how the pass rate changes going forward, and it will be especially interesting when those completing ACGME-accredited fellowships start taking the exam.

One bit of advice I definitely give to any physician who meets the practice pathway qualifications (or can do so by 2017) is to sit for the exam before the end of grandfathering period. After that time, the only way to become certified in the subspecialty will be to complete a two-year, on-site, ACGME-accredited fellowship. While we were excited to be the third program nationally to launch a fellowship at OHSU, it will be a challenge for those who are mid-career, with jobs, family, and/or geographical roots, to up and move to become board-certified.

But starting in 2018, board certification for physicians not able to pursue fellowships will become much more difficult. There are many categories of individuals for whom getting certified in the subspecialty after the grandfathering period will be a challenge:
  • Those who are mid-career - I have written in the past that the age range of OHSU online informatics students, including physicians, is spread almost evenly across all ages up to 65. Many physicians transition into informatics during the course of their careers, and not necessarily at the start.
  • Those pursuing research training in informatics, such as an NLM fellowship or, in the case of some of our current students, in an MD/PhD program (and will not finish their residency until after the grandfathering period ends) - Why must these individuals also need to pursue an ACGME-accredited clinical fellowship to be eligible for the board exam?
  • Those who already have had long medical training experiences, such as subspecialists with six or more years of training - Would such individuals want to do two additional years of informatics when, as I recently pointed out, it might be an ideal experience for them to overlay informatics and their subspecialty training?
Fortunately, one option for physicians who are not eligible for board exam will be the Advanced Health Informatics Certification being developed by AMIA. This certification will be available to all practitioners of informatics trained at the master's level and higher. It will also provide a pathway for physicians who are not eligible for the board certification pathway. I am looking forward to AMIA releasing its detailed plans for this certification, not only for these physicians but also other practitioners of informatics.

However, I also hold out hope for the ideal situation for physician-informaticians, which in my opinion will be our own specialty. The work of informatics carried out by physicians is unique and not really dependent on their initial clinical specialty (or lack of one at all). I still believe that robust training is required to be an informatician; I just don't believe it needs to be a two-year, in-residence experience. An online master's degree or something equivalent, with a good deal of experiential learning in real-world settings, should be an option. The lack of these sorts of options will keep many talented physicians from joining the field. Such training would also be consistent with the 21st century knowledge workforce that will involve many career transitions over one's working lifetime.

Saturday, February 13, 2016

Health IT and the Limits to Analogies

Many who write and talk about health IT, including myself, are fond of using analogies. One of the most common analogies that we use is that of the banking industry. I have noted that I can insert my Wells Fargo ATM card into just about any ATM in the world and receive out local currency. This is all made possible by a standard adopted worldwide by the banking industry. Of course, there is another reason for banking interoperability that does not exist in healthcare, which is that the financial incentives are all aligned. Each time we make an ATM transaction, a fee goes to both the bank that owns the ATM and (if the machine is owned by a bank different from our own) our bank. While most of us grumble about ATM fees, we usually pay them, not only because we have to, but also because of the convenience.

Another common analogy we use for how health IT could be better is to discuss the aviation industry. There is no question that healthcare could learn more from not only the IT of the aviation industry, but also the relationships between all the players who insure that planes take off and land safely [1]. With regards to the IT of aviation, there is definitely more human factors and usability analysis that go into the design of cockpit displays for flying these complex machines than is done in healthcare.

An addition to the analogy list I hear with increasing frequency is the smartphone. In particular, many ask, why can’t the electronic health record (EHR) be as simple as a smartphone? Again, there is much to learn from the simplicity and ease of use of smartphones, especially their organization as allowing “substitutable” apps on top of a common data store and set of features, such as GPS [2]. However, there are also limitations to the smartphone analogy. First, the uses of the EHR are much more complex than most smartphone apps. There is a much larger quantity and diversity of data in the patient’s record. Second, the functions of viewing results, placing orders, and other actions are much more complex than our interactions with simple apps.

I look at my own smartphone usage and note that I spend a great deal of time (probably too much) using it. But there are many things I do with my laptop that I cannot do with my smartphone. For example, my phone is fine for reading email and typing simple replies. However, composing longer replies or working with attachments is not feasible, at least for me, on my phone. Likewise, writing documents, creating presentations, and carrying out other work requiring more than a small screen is also not possible on my phone with its limited screen, keyboard, and file storage capabilities.

While a key challenge of informatics is to make the EHR simpler and easier to use, it will never approach the simplicity of a highly focused smartphone app. Analogies can be helpful in elucidating problems, but we also must recognize their limitations.

References

1. Pronovost, PJ, Goeschel, CA, et al. (2009). Reducing health care hazards: lessons from the commercial aviation safety team. Health Affairs. 28: w479-w489.
2. Mandl, KD, Mandel, JC, et al. (2012). The SMART Platform: early experience enabling substitutable applications for electronic health records. Journal of the American Medical Informatics Association. 19: 597-603.

Monday, February 1, 2016

60 Years of Informatics: In the Context of Data Science

Like many academic health science universities, my institution has undertaken a planning process around data science. In the process of figuring how to merge our various data-related silos, we tried to look at what other universities were doing. One high-profile effort has been launched at the University of Michigan, and the formation of their program and those of others inspired a statistician, David Donoho, to look at data science from the purview of his field 50 years after famed statistician John Turkey had called for reformulation of the discipline into a science of learning from data. Donoho’s resulting paper [1] motivated me to look at data science from the purview of my field, biomedical and health informatics.

Statistics has of course been around for centuries, although this author drew from an event 50 years ago, a lecture by George Tukey. The informatics field has not been in existence for as many centuries, but one summary of its history by Fourman credits the origin of the term to Philip Dreyfus in 1962 [2]. However, the Wikipedia entry for informatics attributes the term to a German computer scientist Karl Steinbuch in 1956. Fourman also notes that the heaviest use of the term informatics comes from its attachment to various biomedical and health terms [2].

If the informatics field is indeed 60 years old, I have been working in it for about half of its existence, since I started my National Library of Medicine (NLM) medical informatics fellowship in 1987. I have certainly devoted a part of my career to raising awareness of the term informatics, making the case for it as a discipline [3]. Clearly the discipline has become recognized, with many academic departments, mostly in health science universities, and a new physician subspecialty devoted to it [4].

And now comes data science. What are we in informatics to make of this new field? Is it the same as informatics? If not, how does it differ? I have written about this before.

Donoho’s paper does offer some interesting insights [1]. I get a kick out of one tongue-in-cheek definition he gives of a data scientist, whom he defines as a “person who is better at statistics than any software engineer and better at software engineering than any statistician.” Perhaps we could substitute informatician for software engineer, i.e., a data scientist is someone who is better at statistics than any informatician and is better at informatics than any statistician?

Donoho does later provide a more serious definition of data science, which is that it is “the science of learning from data; it studies the methods involved in the analysis and processing of data and proposes technology to improve methods in an evidence-based manner.” He goes on to further note, “the scope and impact of this science will expand enormously in coming decades as scientific data and data about science itself become ubiquitously available.”

Donoho goes on to note six key aspects (he calls them “divisions” of “greater data science”) that I believe further serve to define the work of the field:
  • Data Exploration and Preparation
  • Data Representation and Transformation
  • Computing with Data
  • Data Modeling
  • Data Visualization and Presentation
  • Science about Data Science
Clearly data is important to informatics. But is it everything? We can being to answer this question by thinking about the activities of informatics where data, at least not “Big Data,” is not central. While I suppose it could be argued that all applications of informatics make use of some amount of data, there are aspects of those applications where data is not the central element. Consider the many complaints that have emerged around the adoption of electronic health records, such as poor usability, impeding of workflow, and even concerns around patient safety [5]. Academic health science leaders can lead the charge in use of data but must do so in the context of a framework that protects the rights of patients, clinicians, and others [6].

Like many informaticians, I do remain enthusiastic for the prospect of the growing quantity of data to advance our understanding of human health and disease, and how to treat the latter better. But I also have some caveats. I have concerns that some data scientists read too much into correlations and associations, especially in the face of so much medical data capture being imprecise, our lack of adoption of standards, and its inaccessibility when not structured well (which can lead us to try to “unscramble eggs”).

It is clear that informatics cannot ignore data science, but our field must also be among the leaders in determining its proper place and usage, especially in health-related areas. We must recognize the overlap as well as appreciate the areas where informatics can be synergistic with data science.

References

1. Donoho, D (2015). 50 years of Data Science. Princeton NJ, Tukey Centennial Workshop. https://dl.dropboxusercontent.com/u/23421017/50YearsDataScience.pdf.
2. Fourman, M (2002). Informatics. In International Encyclopedia of Information and Library Science, 2nd Edition. J. Feather and P. Sturges. London, England, Routledge: 237-244.
3. Hersh, W (2009). A stimulus to define informatics and health information technology. BMC Medical Informatics & Decision Making. 9: 24. http://www.biomedcentral.com/1472-6947/9/24/.
4. Detmer, DE and Shortliffe, EH (2014). Clinical informatics: prospects for a new medical subspecialty. Journal of the American Medical Association. 311: 2067-2068.
5. Rosenbaum, L (2015). Transitional chaos or enduring harm? The EHR and the disruption of medicine. New England Journal of Medicine. 373: 1585-1588.
6. Koster, J, Stewart, E, et al. (2016). Health care transformation: a strategy rooted in data and analytics. Academic Medicine. Epub ahead of print.

Monday, January 25, 2016

Biomedical Data Science Needs Measures of Information Density and Value

I wrote recently that one of my concerns for data science is the Big Data over-emphasis on one of its four Vs, namely volume. Since then, I was emailing with Dr. Shaun Grannis and other colleagues from the Indiana Health Information Exchange (IHIE). I asked them about size of their data for near 6 billion clinical observations from the 17 million patients in their system. I was somewhat surprised to hear that the structured data only takes up 26 terabytes. I joked that I almost have that much disk storage lying around my office and home. That is a huge amount of data, but some in data science seem to imply that data sizes that do not seem to start with at least “peta-” are somehow not real data science.

Of course, imaging and other binary data add much more to the size of the IHIE data, as will the intermediate products of various processing that are carried out when doing analysis. But it is clear that the information “density” or “value” contained in that 26 terabytes is probably much higher than a comparable amount of binary (e.g., imaging, genome, etc.) data. This leads me to wonder whether we should be thinking about how we might measure the density or value of different types of biomedical and health information, especially if we are talking about the Vs of Big Data.

The measurement of information is decades old. Its origin is attributed to Shannon and Weaver from a seminal publication in 1949 [1]. They defined information as the number of forms a message could take. As such, a coin flip has 2 bits of information (heads or tails), a single die has 6 bits, and a letter in the English language has 26 bits. This measure is of course simplistic in that it assumes the value of each form in the message is equal. For this reason, others such as Bar Hillel and Carnap began adding semantics (meaning) that, among other things, allowed differing values for each form [2].

We can certainly think of plenty of biomedical examples where the number of different forms that data can take yields widely divergent value of the information. For example, the human genome contains 3 billion nucleotide pairs, each of which can take 4 forms. Uncompressed, and not accounting for the fact a large proportion is identical across all humans [3], this genome by Shannon and Weaver’s measure would have 12 billion bits of information. The real picture of human genomic variation is more complex (such as through copy number variations), and the point is that there is less information density in the huge amount of data in a genome than in, say, a short clinical fact, such as a physical exam finding or a diagnosis.

By the same token, images also have different information density than clinical facts. This is especially so as the resolution of digital images continues to increase. There is certainly value in higher-resolution images, but there are also diminishing returns in terms of the information value. Doubling or quintupling or any other increase of pixels or their depth will create more information as measured by Shannon and Weaver’s formula but not necessarily provide more value of that information.

Even clinical data may have diminishing returns based on its size. Some interesting work from OHSU faculty Nicole Weiskopf and colleagues demonstrates an obvious finding but one that has numerous implications for secondary use of clinical data, which is that sicker patients have more data in the electronic health record (EHR) [4-5]. The importance of this is that sicker patients may be “oversampled” in clinical data sets and thus skew secondary analysis by over-representing patients who have received more healthcare.

There are a number of implications for increasing volumes of data that we must take into consideration, especially when using such data for purposes for which it was not collected. This is probably true for any Big Data endeavor, where the data may be biased by the frequency and depth of its measuring. The EHR in particular is not a continuous sampling of a patient’s course, but rather represents periods of sampling that course. With the EHR there is also the challenge that different individual clinicians collect and enter data differently.

Another implication of data volumes is its impact on statistical significance testing. This is one form of what many criticize in science as “p-hacking,” where researchers modify the presentation of their data in order to achieve a certain value for the p statistic that measures the likelihood that differences are not due to chance [6]. Most researchers are well aware that their samples must be of sufficient size in order to achieve the statistical power to attain a significant difference. However, on the flip side, it is very easy to obtain a p value that shows small, perhaps meaningless, differences are statistically significant when one has very large quantities of data.

The bottom line is that as we think about using data science, certainly in biomedicine and health, and the development of information systems to store and analyze it, we must consider the value of information. Just because data is big does not mean it is more important than when data is small. Data science needs to focus on all types and sizes of data.

References

1. Shannon, CE and Weaver, W (1949). The Mathematical Theory of Communication. Urbana, IL, University of Illinois Press.
2/ Bar-Hillel, Y and Carnap, R (1953). Semantic information. British Journal for the Philosophy of Science. 4: 147-157.
3. Abecasis, GR, Auton, A, et al. (2012). An integrated map of genetic variation from 1,092 human genomes. Nature. 491: 56-65.
4. Weiskopf, NG, Rusanov, A, et al. (2013). Sick patients have more data: the non-random completeness of electronic health records. AMIA Annual Symposium Proceedings 2013, Washington, DC. 1472-1477.
5. Rusanov, A, Weiskopf, NG, et al. (2014). Hidden in plain sight: bias towards sick patients when sampling patients with sufficient electronic health record data for research. BMC Medical Informatics & Decision Making. 14: 51. http://bmcmedinformdecismak.biomedcentral.com/articles/10.1186/1472-6947-14-51.
6. Head, ML, Holman, L, et al. (2015). The extent and consequences of p-hacking in science. PLoS Biology. 13: e1002106. http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1002106.

Monday, January 18, 2016

Meaningful Use Ending? Yes and No

The health information technology (HIT) world was shaken to its core last week by Andy Slavitt, Acting Administrator of the Centers for Medicare & Medicaid Services (CMS), who announced that the CMS Meaningful Use (MU) program was over. More precisely, he stated, “The Meaningful Use program as it has existed, will now be effectively over and replaced with something better.” He tweeted this as well, and it was retweeted by Dr. Karen DeSalvo, Director of the Office of the National Coordinator for Health IT (ONC). A transcript of his comments were posted in The CMS Blog. The health IT media was quick to pick up on his remarks.

Does this mean that eligible professionals and hospitals will no longer need to meet MU criteria to get CMS incentive dollars or avoid penalties. Sort of. Months ago, in one of the few bipartisan moves of the current Congress, passed the Medicare Access and CHIP Reauthorization Act (MACRA) legislation [1]. This legislation is best known as the “doc fix” because it fixed the problem of the old sustainable growth rate (SGR) formula of Medicare that threatened to substantially decrease physician reimbursement under Medicare.

Another part of the MACRA legislation addressed some criticisms of the various Medicare incentive programs, such as their multiple number, i.e., not only MU, but also the Physician Quality Reporting System (PQRS) and the Value-Based Payment Modifier (VM). These will all be rolled into a single Merit-based Incentive Payments (MIPS) program, which will start to assess penalties in a graduated way, from 4% in 2019 up to 9% in 2022. After 2019, CMS will also provide another pathway to incentives via Alternative Payment Models (APMs), such as through accountable care organizations (ACOs).

The MIPS program will consist of four categories of measures (divided among percentages): quality (30%), resource use (30%) , clinical practice improvement activities (15%), and MU of certified EHR technology (25%). The details of these are under development by CMS, but it is clear that within MIPS, MU will be part of what eligible hospitals and eligible professionals will need to achieve to qualify for Medicare incentives and avoid penalties. As of right now, that includes MU, including Stage 3 [2]. What any new approach will look like going forward is not known. Stay tuned!

Some suggestions for improving MU going forward from John Halamka speaking on behalf of about 30 healthcare delivery organizations and Peter Basch and Thomson Kuhn speaking on their own but as leaders from the American College of Physicians.

Both Basch and Kuhn as well as Halamka deem MU Stage 1 a success in terms of achieving widespread adoption, but note it is time to move beyond the functional-use measures of MU. They call for Stage 3 as it is currently planned to be abandoned and also note how the highly persecutive approach stifles innovation by clinicians and boxes in the work of EHE developers. Basch and Kuhn go farther in terms of making recommendations. They call for a reconfiguration of MU within MIPS, with the elimination of functional-use measure thresholds (e.g., 80% of patients with problem lists or use of five clinical decision support rules), judicious use of non-threshold functional-use measures, practical interoperability that allows the delivery of high-quality care, more flexible engagement with patients, more innovative approaches to participating in and measuring quality initiatives. A final call they make is for continuing medical education within the domains of health IT so that physicians (and others) can learn how to deliver the best care using IT.

When writing about the present situation of the HITECH Act, I often harken back to what I wrote when it was first unveiled: "This is a defining moment for the informatics field. Never before has such money and attention been lavished on it. HITECH provides a clear challenge for the field to 'get it right.' It will be interesting to look back on this time in the years ahead and see what worked and did not work. Whatever does happen, it is clear that informatics lives in a HITECH world now." It is now time to move on from HITECH and MU to a more sustaining health IT that meets the needs of the healthcare system going forward.

References

1. Doherty, RB (2015). Goodbye, sustainable growth rate—hello, merit-based incentive payment system. Annals of Internal Medicine. 163: 138-139.
2. O'Neill, T (2015). Primer: EHR Stage 3 Meaningful Use Requirements. Washington, DC, American Action Forum. http://americanactionforum.org/uploads/files/research/2015-10-20_Primer_Stage_3_Meaningful_Use_Final_Rule_%28CH_edits%29_ej_mg_sh_....pdf.

Wednesday, January 13, 2016

A Tale of Childbirth in Two Countries and Some Teachable Moments

As the father of two healthy adult children who were not born in hospitals, as well as being married to a certified nurse midwife (CNM), I take great interest in childbirth, especially its evidence-based aspects. Both of my children were born in out-of-hospital birth centers, where I was the only physician present, and was in no way there in a physician capacity. Both were delivered by CNMs. I always marvel at how pleasant the experience was relative to the eight deliveries I performed as a medical student in an academic hospital in the 1980s.

Two recent studies provide a number of teachable moments concerning evidence-based care. Both studies asked roughly the same question, looking at the risk of perinatal mortality for planned in-hospital vs. out-of-hospital births. One study from Ontario, Canada found no differences in neonatal mortality for planned out-of-hospital births by midwives compared to planned in-hospital births [1]. But another study from Oregon found that there was a higher risk of neonatal mortality in planned out-of-hospital births (3.9 deaths compared to 1.8 deaths per 1000 deliveries) [2]. This study also found a much higher rate of cesarean section (C-section) for in-hospital births (28.1% versus 6.2%). Caesarian sections are associated with a variety of short and longer term patient complications.

What are the teachable moments? Certainly one is that out-of-hospital birth is very safe, especially in a healthcare system when care among midwives and physicians is highly coordinated, as occurs in Canada and most other developed countries. That is not the case in the United States, and it is likely that some complications in Oregon were a result of that lack of coordination. But even in the United States, at least in Oregon, out-of-hospital birth is relatively safe.

Another teachable moment concerns relative versus absolute risk. When stated as a relative risk ratio, the difference in mortality in Oregon was 2.2-fold higher. But neonatal mortality is an extremely rare event. As such, the absolute risk difference between in-hospital and out-of-hospital birth was only 2.1 deaths per 1000 deliveries. This was accompanied by a relative risk ratio for C-section that was 4.5-fold higher and which also had a substantially higher absolute risk difference of 21.9% of all deliveries.

An additional teachable moment concerns some important issues around data. This study was made possible because several years ago, the state of Oregon added a question to birth certificates that asked all women who had an in-hospital delivery, “Did you go into labor planning to deliver at home or at a freestanding birthing center?” This enabled the researchers to determine planned vs. unplanned out-of-hospital births and thus made this study possible. Data does not magically just appear; we have to determine what we want to collect, make plans to collect it, and determine its completeness and validity.

These studies also raise the question of whether the difference in neonatal mortality in Oregon could be ameliorated by better care coordination. The Oregon study also raises the question of whether moving deliveries from hospitals to other settings would reduce the C-section rate. Hopefully these and other questions will be answerable in the future. I certainly hope it may lead to more families being able to experience the pleasant deliveries I was able to have with my children.

References

1. Hutton, EK, Cappelletti, A, et al. (2015). Outcomes associated with planned place of birth among women with low-risk pregnancies. Canadian Medical Association Journal. Epub ahead of print.
2. Snowden, JM, Tilden, EL, et al. (2015). Planned out-of-hospital birth and birth outcomes. New England Journal of Medicine. 373: 2642-2653.

Saturday, January 9, 2016

My Changing Relationship With Photography

I imagine that like many people, my relationship with photography has changed over the years. In my current life, I do like to capture many aspects of my life with pictures. But my approach to photography has changed over the years.

I was not always a big-time photographer. While I certainly have plenty of pictures of my children as well as major events in my life, I did very little to capture much more beyond that in my earlier adulthood.

That all changed with the advent of digital photography. Even with the early digital cameras that took pictures that were (compared to today) of poorer quality and had extremely limited memory (so you could not store many pictures on the camera), I took to electronic photos. No doubt the convenience and instant gratification of seeing the results right away and no longer having to send film for developing played a huge role.

Despite my proclivity for digital photography, I have never invested in any high-end cameras. I have definitely preferred the convenience of point-and-shoot cameras that were quick to turn on, easy to take pictures, and simple to transfer them to a computer.

Along the way I also for the most part stopped printing pictures. While I do occasionally have a reason to print a photo, for the most part I view my pictures on my computer or mobile devices.

In 2014 I made the plunge into a somewhat higher-end camera, a Sony Alpha 6000. These “mirrorless” cameras supposedly approach digital single-lens reflex (DSLR) cameras in quality, with a more compact size (though not able to fit in a pocket). This particular camera also gave me, for the first time ever, a changeable (zoom) lens. It is definitely a nice camera and takes great pictures, especially when a zoom lens is preferable or lighting conditions are suboptimal.

About the same time, however, I had upgraded my smartphone to an iPhone 6. The newer cameras on these modern smartphones also take excellent pictures, especially in decent light and when not requiring any sort of zoom. Of course a major convenience is that they can be carried in a pocket (and also serve as phone, Internet access device, music player, and more). Another critical benefit of smartphone cameras is the convenience of posting photos on Facebook, which is something I do frequently. I can also carry it when I go running if there is a need for having a camera.


I now take the overwhelming majority of my pictures with my smartphone. The quality of most of them is exemplary, and when married with the convenience of being part of a device that easily fits in my pocket, my Sony camera is reserved mostly for special occasions. Those occasions are still important, so I will likely never part with a camera better than that on my smartphone, although who knows what technology for the latter may develop in the future to change even that.

Thursday, December 31, 2015

Annual Reflections at the End of 2015

As regular readers of this blog know, I traditionally end each year with a posting reflecting back on the past year. While this year has been another great success for myself and our informatics program at Oregon Health & Science University (OHSU), it has been somewhat of a transitional year for the informatics field. Many of the new and exciting initiatives in the informatics field from recent years are no longer novel, with some now settling into “midlife” and others being called out for retirement.

One program settling into midlife, although being called out for retirement by many, has been the Health Information Technology for Economic and Clinical Health (HITECH) Act. The launching of this blog, and indeed the catapult to much larger visibility of the informatics field, owes a great deal to HITECH. There is no question that HITECH has succeeded on some levels, at least in terms of increasing electronic health record (EHR) adoption, as I have noted before. A recent report from the Commonwealth Fund confirms what statistics from the Office of the National Coordinator for Health IT (ONC) show: the US is no longer a world laggard in health IT and is in some ways a global leader [1].

But there is no question that not all with HITECH has gone well. Despite the widespread adoption of EHRs, they are still very imperfect [2]. At best, they impede clinician workflow and at worst, they cause some of the safety problems they have been touted to rectify. And one vision has clearly not been achieved, which is interoperable data across systems, even those from the same vendor [3]. Going forward, the informatics field must provide leadership to guide the best use of EHRs and related systems, which is spelled out excellently in the AMIA EHR-2020 Task Force white paper [4].

Another interesting happening, perhaps related to health IT achieving midlife, is that the quantity of health IT blogging seems to be tapering off. In this blog for example, I had fewer posts this year than any since the first year I started the blog. The same is true for a number of other well-known health IT bloggers, such as Keith Boone and John Halamka. I do not view this as necessarily a bad thing, but perhaps just an indicator that some of the formerly novel aspects of informatics are reaching maturity, and there is less to say on a day-to-day basis.

Also a continuing happening this year was the continued growth of data science, and confusion as to its relationship to informatics. Informaticians are not the only ones expressing confusion where they belong in this new field; statisticians are feeling the same [5]. Nonetheless, there is no question that data and learning from it will drive many scientific fields going forward.

I would also like to call out some other year-end posts from some other bloggers, namely John Halamka, for recapping 2015 overall plus adding some focus on security and looking ahead to 2016, and the folks at HISTalk, who have a comprehensive list of 2015 top stories and 2016 predictions.

On a personal and program level, this year had a number of achievements. I was honored to be bestowed the HIMSS Physician IT Leadership Award. I was also awarded a new grant to update the ONC Health IT Curriculum. On a program level, the OHSU Department of Medical Informatics & Clinical Epidemiology (DMICE) launched its new clinical informatics fellowship and continued its mutli-faceted success in its major missions of research and education.

Looking ahead to 2016, there are plenty of new projects and other activities to keep myself and our department busy. It will be interesting to see how HITECH fares and how the critical need for data interoperability evolves. And of course, new opportunities will emerge for myself and DMICE, many of which cannot even be foreseen now.

References

1. Osborn, R, Moulds, D, et al. (2015). Primary care physicians in ten countries report challenges caring for patients with complex health needs. Health Affairs. 34: 2104-2112.
2. Rosenbaum, L (2015). Transitional chaos or enduring harm? The EHR and the disruption of medicine. New England Journal of Medicine. 373: 1585-1588.
3. Anonymous (2015). Connecting Health and Care for the Nation: A Shared Nationwide Interoperability Roadmap version 1.0 (Roadmap). Washington, DC, Department of Health and Human Services. https://www.healthit.gov/sites/default/files/hie-interoperability/nationwide-interoperability-roadmap-final-version-1.0.pdf.
4. Payne, TH, Corley, S, et al. (2015). Report of the AMIA EHR-2020 Task Force on the status and future direction of EHRs. Journal of the American Medical Informatics Association. 22: 1102-1110.
5. Donoho, D (2015). 50 years of Data Science. Princeton NJ, Tukey Centennial Workshop. https://dl.dropboxusercontent.com/u/23421017/50YearsDataScience.pdf.

Wednesday, December 30, 2015

Volume is Only One of the Four "V"s of Big Data, Especially for the Right Data

One widely accepted definition of Big Data is that it entails four “V”s: volume, velocity, variety, and veracity. In other words, Big Data is defined by there being a great deal of it (volume), coming at us rapidly and continuously (velocity), taking many different forms and types (variety), and originating from trustworthy sources (veracity). Among some people, however, there seems to be more focus on one of the Vs above all others, namely volume. I suppose that is not surprising, given that the adjective qualifying the noun head in Big Data is one that describes size.

However, as I and others have written over the years, there are many aspects of data that are just as important as its quantity. Even worse, I have heard many people imply in their statements about data science that you cannot do real data science without massive amounts of data, in turn requiring massive amounts of storage capacity and computer power (also costing much money).

Make no mistake, we do need to consider the volume aspects of data when discussing data science. But we must not lose in the discussion what we hope to accomplish with the data, which one writer refers to as the fifth V of Big Data, namely value [1]. Sometimes value emanates from harnessing the size of a data set, but other times the veracity or variety take on more importance.

I have written about the importance of value as well, noting that meaningless correlations with large amounts of data do not really mean much of anything, and that data scientists must also understand basic research principles, such as causality. So yes, let us prepare for a future where we leverage Big Data to improve health, biomedicine, and other important societal needs, but we also need to remember that we do not always need massive amounts of data, especially that whose veracity we may not know, to derive other value. Perhaps akin to the “rights” of clinical decision support [2], the best data science is more about having access to the right data using the right amount of data at the right time.

References

1. Marr, B (2015). Why only one of the 5 Vs of big data really matters. IBM Big Data & Analytics Hub. http://www.ibmbigdatahub.com/blog/why-only-one-5-vs-big-data-really-matters.
2. Osheroff, JA, Teich, JM, et al. (2012). Improving Outcomes with Clinical Decision Support: An Implementer's Guide, Second Edition. Chicago, IL, Healthcare Information Management Systems Society.

Monday, December 21, 2015

New NIH Biosketch Allows Better Documentation of Contributions to Science

One of the most important documents for a US-based biomedical researcher is the National Institutes of Health (NIH) Biosketch. This short document summarizes the accomplishments of a scientist apply for an NIH grant, listing his or her job positions, educational history, a summary of key publications, and a listing of current grant funding. The NIH Biosketch is also often used as a summary of one’s larger curriculum vitae (CV).

NIH has tweaked the Biosketch over the years, and the most recent update provides an excellent approach that allows researchers to not just summarize their most prominent publications, but also to give a statement about them in the context of the individual's contributions to science. For each contribution, he or she can provide up to four key publications for each. I enjoyed the exercise of updating my Biosketch to the new form, and thought it would be worthwhile to reproduce the scientific contributions and key publications here.

1. My initial research focused on the development and implementation of information retrieval (IR, also called search) systems in biomedicine and health. I experimented with concept-based approaches to indexing and retrieval of knowledge-based information. Subsequently, I found that methods for evaluation systems were inadequate, and developed an interest in new approaches to evaluation. My interests in search have also evolved with the emergence of new content for retrieval, such as medical images and electronic health record data, especially textual notes in the latter.
  • Hersh WR, Greenes RA, SAPHIRE: an information retrieval system featuring concept matching, automatic indexing, probabilistic retrieval, and hierarchical relationships, Computers and Biomedical Research, 1990, 23: 410-425.
  • Hersh WR, Crabtree MK, Hickam DH, Sacherek L, Friedman CP, Tidmarsh P, Moesbaek C, Kraemer D, Factors associated with success for searching MEDLINE and applying evidence to answer clinical questions, Journal of the American Medical Informatics Association, 2002, 9: 283-293. PMC344588.
  • Hersh W, Kalpathy-Cramer J, Müller H, The ImageCLEFmed medical image retrieval task test collection, Journal of Digital Imaging, 2009, 22: 648-655.
  • Hersh W, Voorhees E, TREC Genomics special issue overview, Information Retrieval, 2009, 12: 1-15.
2. My interest work in IR has converged with another interest in the secondary use of clinical (especially electronic health record) data. I have made contributions not only in attempting to leverage such data, but also addressing caveats and recommendations for its use.
  • Voorhees E, Hersh W, Overview of the TREC 2012 Medical Records Track, The 21st Text Retrieval Conference - TREC 2012.
  • Edinger T, Cohen AM, Bedrick S, Ambert K, Hersh W, Barriers to retrieving patient information from electronic health record data: failure analysis from the TREC Medical Records Track, Proceedings of the AMIA 2012 Annual Symposium, 2012, 180-188, PMC3540501.
  • Hersh WR, Weiner MG, Embi PJ, Logan JR, Payne PR, Bernstam EV, Lehmann HP, Hripcsak G, Hartzog TH, Cimino JJ, Saltz JH, Caveats for the use of operational electronic health record data in comparative effectiveness research, Medical Care, 2013, 51(Suppl 3): S30-S37. PMC3748381.
  • Hersh WR, Cimino JJ, Payne PR, Embi PJ, Logan JR, Weiner MG, Bernstam EV, Lehmann HP, Hripcsak G, Hartzog TH, Saltz JH, Recommendations for the use of operational electronic health record data in comparative effectiveness research, eGEMs (Generating Evidence & Methods to improve patient outcomes), 2013, 1:14.
3. I have also made contributions in conducting systematic reviews of evaluative research of informatics technologies. These reviews can be challenging because many evaluations use weak evaluation methodologies, in part because these technologies are tools rather than typical medical tests or treatments.
  • Hersh WR, Hickam DH, How well do physicians use electronic information retrieval systems? A framework for investigation and systematic review, Journal of the American Medical Association, 1998, 280: 1347-1352.
  • Hersh WR, Hickam DH, Severance SM, Dana TL, Krages KP, Helfand M, Diagnosis, access, and outcomes: update of a systematic review on telemedicine services, Journal of Telemedicine and Telecare, 2006, 12(Supp 2): 3-31.
  • Stanfill MH, Williams M, Fenton SH, Jenders R, Hersh W, A systematic review of automated clinical coding and classification systems, Journal of the American Medical Informatics Association, 2010, 17: 646-651. PMC3000748.
  • Hersh W, Totten A, Eden K, Devine B, Gorman P, Kassakian S, Woods SS, Daeges M, Pappas M, McDonagh MS, Outcomes from health information exchange: systematic review and future research needs, JMIR Medical Informatics, 2015, 3(4): e39.
4. Being the leader of a major biomedical informatics educational program, I have also carried out research characterizing the informatics professional workforce. My study on the need for health IT professionals played a role in workforce development being a component of the Health Information Technology for Clinical and Economic Health (HITECH) Act of the American Recovery and Reinvestment Act (ARRA).
  • Hersh W, Who are the informaticians? What we know and should know, Journal of the American Medical Informatics Association, 2006, 13: 166-170. PMC1447543.
  • Hersh W, Wright A, What workforce is needed to implement the health information technology agenda? Analysis from the HIMSS Analytics™ Database, Proceedings of the AMIA 2008 Annual Symposium, 2008, 303-307. PMC2656033.
  • Hersh W, The health information technology workforce: estimations of demands and a framework for requirements, Applied Clinical Informatics, 2010, 1: 197-212. PMC3632279.
  • Hersh WR, Margolis A, Quirós F, Otero P, Building a health informatics workforce in developing countries, Health Affairs, 2010, 29: 274-277.
5. Also as a result of being an educational leader, I have carried out evaluation of educational programs in informatics, including those using distance learning technologies.
  • Hersh W, Williamson J, Educating 10,000 informaticians by 2010: the AMIA 10x10 program, International Journal of Medical Informatics, 2007, 76: 377-382.
  • Hersh WR, A stimulus to define informatics and health information technology, BMC Medical Informatics and Decision Making, 2009, 9: 24.
  • Otero P, Hersh W, Luna D, Quirós F, Translation, implementation and evaluation of a medical informatics distance-learning course for Latin America, Methods of Information in Medicine, 2010, 49: 310-315.
  • Mohan V, Abbott P, Acteson S, Berner ES, Devlin C, Hammond WE, Kukafka R, Hersh W, Design and evaluation of the ONC health information technology curriculum, Journal of the American Medical Informatics Association, 2014, 21: 509-516.
NIH now also allows scientists to create a complete list of published work in the MyBibliography section of the NCBI Web site.

Tuesday, December 15, 2015

The Evidence Base for Health Information Exchange

One of my major projects over the last couple years has been a systematic review of the research that has been conducted on health information exchange (HIE). I wrote about this project when it first started and when our protocol for the review was posted for public comment. The report was funded by the Evidence-Based Practice Centers Program of the Agency for Healthcare Research and Quality (AHRQ). While the review itself has been done for several months, we have been finalizing the report and publications derived from it since then. I am pleased to report that both the complete report [1] plus a paper reporting on the outcomes from studies of HIE [2] have now been published. There will be some additional papers on other aspects of the report as well as a book chapter summarizing the report to be published next year [3].

This report has certainly given me the opportunity to reflect over the last couple years of the state of HIE and the interoperability required to support it. The major finding of the report echoes findings of a similar couple of systematic reviews I led on the topic of telemedicine published in 2001 [4] and 2006 [5], which is that the breadth and quality of the research are limited. There is no question that performing research on HIE is difficult. After all, HIE is not a test or a treatment, but rather a tool that facilitates other aspects of healthcare. Nonetheless, the research base for HIE is limited, and should be improved if we want to discern it benefits and optimal use. The paper provides our recommendations for improving research on HIE outcomes going forward [2].

Our report also gives us an opportunity to think about some of the larger issues around the current role and future directions of HIE. If I had to lament about HIE, I would say that it is an unfortunate requirement at this time for us to need so many different organizations (135 according to the last eHI annual survey of them [6]) devoted to HIE. In the ideal world, there would be no need for HIE organizations, but instead, there would be sufficient interoperability of systems, along with rules and regulations, to allow information to flow seamlessly between appropriate parts of the healthcare system. For example, a physician in his or her office could seamlessly transmit a consultation, receive laboratory results or a discharge summary, or notify a public health department of a reportable event without requiring an HIE entity to facilitate those activities. The information transmitted would be formatted into some standardized form and sent securely to an authenticated site, all facilitated by standard protocols used by the entire industry.

Hopefully the new emphasis of ONC on interoperability [7] and the underlying standards required [8] will facilitate more seamless HIE. While many have argued that the criteria for meaningful use should have placed more emphasis on secure and standardized information exchange rather than specific EHR functionality, such as clinical decision support or specific quality measures, that is all now proverbial water under the bridge. I am certain everyone agrees that we need to focus on seamlessly interoperable health IT going forward. I also hope in the process that robust research is carried out, not only to assess the value of HIE but also determine the best ways to implement it.

An interesting side note to this report is an episode related to another systematic review on HIE that was published in late 2014 [9]. One of our competitors for the contract that was awarded by AHRQ to our institution went out and found another source for funding to carry out a review. Not only did they perform a review that was reduced in scope from our review (it omitted public health and any type of HIE outside the US), but they were also able to bypass all of the processes that AHRQ has to insure the systematic reviews it funds have stakeholder engagement, public comment, and broad peer review. As such, the other group was able to complete their review well in advance of ours and get it published in a very high profile journal, Annals of Internal Medicine. That journal publishes a good number of AHRQ-funded systematic reviews, but understandably did not want to publish ours after they had already published another systematic review on the topic of HIE. While I have no problems with science being competitive in terms of accolades going to the first to publish, I do find it disappointing that another group basically duplicated our review and short-circuited the usual processes of AHRQ.

References

1. Hersh W, Totten A, Eden K, Devine B, Gorman P, Kassakian S, Woods SS, Daeges M, Pappas M, McDonagh MS. Health Information Exchange. Evidence Report/Technology Assessment No. 220. AHRQ Publication No. 15(16)-E002-EF. Rockville, MD: Agency for Healthcare Research and Quality; 2015. http://www.effectivehealthcare.ahrq.gov/ehc/products/572/2154/health-information-exchange-report-151201.pdf.
2. Hersh, WR, Totten, AM, et al. (2015). Outcomes from health information exchange: systematic review and future research needs. JMIR Medical Informatics, 3(4):e39. http://medinform.jmir.org/2015/4/e39/.
3. Hersh, WR, Totten, AM, et al. (2016). The Evidence Base for Health Information Exchange. In Health Information Exchange: Navigating and Managing a Network of Health Information Systems. B. Dixon. Amsterdam, Netherlands, Elsevier, in press.
4. Hersh, WR, Helfand, M, et al. (2001). Clinical outcomes resulting from telemedicine interventions: a systematic review. BMC Medical Informatics and Decision Making. 1: 5. http://www.biomedcentral.com/1472-6947/1/5.
5. Hersh, WR, Hickam, DH, et al. (2006). Diagnosis, access, and outcomes: update of a systematic review on telemedicine services. Journal of Telemedicine & Telecare. 12(Supp 2): 3-31.
6. Anonymous (2014). 2014 eHI Data Exchange Survey Key Findings. Washington, DC, eHealth Initiative, http://visit.medicity.com/rs/aetnainc/images/2014%20eHI%20Data%20Exchange%20Survey%20Results.pdf.
7. Anonymous (2015). Connecting Health and Care for the Nation: A Shared Nationwide Interoperability Roadmap version 1.0 (Roadmap). Washington, DC, Department of Health and Human Services. https://www.healthit.gov/sites/default/files/hie-interoperability/nationwide-interoperability-roadmap-final-version-1.0.pdf.
8. Anonymous (2015). Draft 2016 Interoperability Standards Advisory. Washington, DC, Department of Health and Human Services. https://www.healthit.gov/standards-advisory/2016.
9. Rudin, RS, Motala, A, et al. (2014). Usage and effect of health information exchange: a systematic review. Annals of Internal Medicine. 161: 803-811.

Sunday, November 15, 2015

Milestones for the AMIA Annual Symposium

One of my favorite events each year is the American Medical Informatics Association (AMIA) Annual Symposium. It is an excellent meeting that brings together presentation of new research, up-to-date summaries of progress in the field, and great social events.

This year marks the 30th consecutive year of my attending the Annual Symposium (which in its early days was called the Symposium on Computer Applications in Medical Care, or SCAMC). The first time I attended the event was in 1986, when I was in my last year of my internal medicine residency. The following year I started my National Library of Medicine (NLM) Postdoctoral Fellowship, the first support of a long line of NLM funding that has supported my career.

A number of AMIA meetings have been particularly memorable over the years. In 1996, I was elected to the American College of Medical Informatics (ACMI), an honorific society that recognizes expertise in biomedical Informatics. In 2005, the AMIA Symposium featured the first in-person session for the end first offering of the 10x10 (“ten by ten”) course, which would go on to become a major AMIA program to provide introductory education in informatics and also reached a new milestone described below. In 2008, I was awarded the Donald A.B. Lindberg Award for Innovation in Informatics. The meeting in 2012 was another special one, as I served as the Scientific Program Chair.

Another important aspect of the meeting for me is the showcase of scientific research and other work emanating from our biomedical and health informatics program at Oregon Health & Science University (OHSU). This year is no exception, and one thing we are particularly proud of this year is OHSU students comprising two of the four finalists for the AMIA 2015 Student Design Challenge. This year’s Student Design Challenge focus is, The Human Side of Big Data – Facilitating Human-Data Interaction. (Postscript: One of the two OHSU finalist teams took first place, the second consecutive year that an OHSU team won the Student Design Challenge!)

Below is a picture of me at this year’s Annual Symposium, enjoying myself as always, especially with this year’s plethora of ribbons everyone is attaching to their name badge.


Another milestone being achieved at this year’s AMIA Annual Symposium is the surpassing of 2000 who have completed the OHSU offering of the AMIA 10x10 course. While the exact number of people completing the course will not be final until a week or two after the meeting, there will be at least 2034 people who have completed the OHSU 10x10 course since its inaugural offering in 2005. We may not have achieved “ten thousand by 2010,” but I am reminded constantly by those who taken the course that it provided an excellent path into the informatics field. My original course has been the flagship course for the program, making up 74% of those who have completed any 10x10 course. The figure below shows the distribution of those completing the different courses of the 10x10 program.


Also of note is that 312 of those completing the OHSU 10x10 offering have done so via so-called “i10x10” offerings based in other countries, which have included Singapore, Saudi Arabia, and Israel. The OHSU 10x10 course also inspired the offering of the course in Spanish from colleagues at Hospital Italiano de Buenos Aires Argentina, starting initially as a direct translation into Spanish but taking on a more Latin American focus over the years.

Because of the interdisciplinary nature of the informatics, I deal with many different academic fields, from medicine to computer science. But there is no question that my heart and home are in informatics. Each year, the AMIA Annual Symposium reaffirms that this is my discipline, and my closest professional colleagues are also among my best friends.