Thursday, April 2, 2020

A Virtual Course in Biomedical and Health Informatics for Medical Students

Medical students from around the US (and the world) have had their education displaced by the Covid-19 pandemic. In many places, there is either desire to keep them away from risk or to preserve personal protecting equipment (PPE) to physicians, nurses, and others directly involved in patient care. As such, the medical education community has worked to identify virtual educational experiences for medical students.

Our contribution is a virtual course in biomedical and health informatics. Readers familiar with my work will recognize the content of this course as emanating from the introductory course in the Oregon Health & Science University (OHSU) Biomedical Informatics Graduate Program. This course is also used in OHSU's offering as part of the American Medical Informatics Association 10x10 ("ten by ten") program. The syllabus for the course details on how medical schools can enroll their students.

We are implementing the course as a 4-week medical student elective, which is awarded 2 credits at OHSU. The course has about 40 hours of lecture, and we anticipate another 40 hours spent on discussion forums, multiple-choice self-assessments for each unit, and optional readings. The course is graded as pass-fail, and passing requires completion of all of 10 units and their self-assessments over the 4 weeks of the course. Due to high demand, we are limiting enrollment to students in US-based allopathic and osteopathic medical schools.

One new offering of the course will begin each week starting Monday, April 6. We will enroll as many students as we have in a single section, and make all of the content available to them for the duration of the 4 weeks. We will make use of the discussion forums built into our LMS to answer questions they have, and raise a few questions for them to answer. At the end of 4 weeks, the course will end, and those who have completed all of the work will receive a passing grade, which we will report back to the contact from each school.

We are asking each medical school handle student enrollment and credit themselves. In other words, we will make the course available through our learning management system (LMS) at OHSU, but we will ask each school to provide us a list of students to enroll and each will get a login to the course. After the course is done, we will report back to the schools on whether each student completed the course or not. We would like for medical schools that participate to handle giving students credit (probably through some sort of self-study elective).

We also prefer that there be a single point of contact for each school with which we communicate. To capture this information, we have created an online survey that asks for the point of contact (please use a university email address), estimated number of students (initially up to 20 per school - we may be able to accommodate more later), and preferred dates (which we may need to change to balance load). After the survey is completed, someone from our staff will contact the schools to work out the details.

In addition, for those interested in less than a full course on informatics, we have an open Web site that provides some of the materials and is being used by some medical schools.

Monday, March 30, 2020

Keeping Evidence-Based in the Midst of a Pandemic

The Covid-19 pandemic requires urgent scientific knowledge about how to best diagnose, treat, and prevent the spread of the SARS-CoV-2 virus. This is at odds with the deliberate nature of evidence-based medicine (EBM), where it is important to use more deliberate methods to discern the best evidence.

Another challenge is to disseminate the results of research as quickly as possible. The availability of preprint servers and other modern Internet tools allow us to publish first and peer review later. But of course that raises worry that inadvertent error or even deliberate falsehoods might taint the quickly expanding evidence base.

How do we achieve a balance? We have already seen the downside of actions moving ahead of the science. Probably the best example of this is the drug hydroxychloroquine. While this drug may prove of value in preventing and treating SARS-CoV-2, it does have significant adverse effects, especially when taken in doses that exceed the normal therapeutic level. In addition, it is a drug whose availability for other diseases it treats, such as lupus, must be maintained for those patients.

Clearly hydroxychloroquine should be studied, but it should ideally be done in as controlled a way as possible, lest we not cause harm or generate false hope. We may not be able to perform classic double-blind, placebo-controlled randomized controlled trials, but we should still enroll and track patients in highly controlled manners. We cannot forget that this is a disease from which the majority of patients fully recover, so we need to make certain that improvements due to any treatment are not just due to normal recovery from the disease. There must be some sort of control group and a diligent follow-up to insure no missing data in control or experimental groups.

In my view, there are a number of critical questions to answer about SARS-CoV-2:
  • How well do tests diagnose active infection with the disease?
  • How well do tests diagnose serum antibodies indicating immunity?
  • What treatments are available for the disease?
  • Are there any preventive treatments for the disease, from drugs to immunizations?
  • What is the best way to prevent spread in the general population?
  • What is the best way to protect healthcare workers treatment patients with the disease?
All of these can be answered with the usual EBM methods of controlled studies that have served us well. They can also be augmented with large-scale data sources from which we are learning to do better observational studies. We can also carry out systematic reviews, with meta-analysis when appropriate, to collate the results of many studies.

Unfortunately, the deliberate pace of EBM must be balanced with the urgency to develop treatments, vaccinations, and methods to curtail spread of the virus. Likewise, the rapid publication of results on preprint servers and other sources must be followed with peer review and collation into systematic reviews and meta-analyses. Hopefully this will give us the best evidence based for treating and preventing this disease.

Wednesday, March 25, 2020

SARS-CoV-2: How Can I Help?

When the history of the global SARS-CoV-2 pandemic is written, the real heroes will be the frontline workers who cared for those in need and/or kept society functioning. This of course includes healthcare workers but also those working as first-responders and in public safety, or in grocery stores, gas stations, telecommunications infrastructure companies, and other essential businesses. They will certainly come off looking much better than political leaders or even “captains” of industry. I hope that whatever economic recovery plan is implemented that these workers will be appropriately rewarded and that society will have a better appreciation for the essential jobs they do.

It is natural for me to wonder how I can best contribute. As I “retired” from clinical practice some time ago, my skills as a clinician are probably not up to the task. However, there are probably some skills I can contribute, and I will consider those options going forward.

Fortunately there are some non-directly clinical contributions I can make, and these are keeping me busy here and now. One is teaching.  While society is first and foremost dealing with the crisis at hand, we cannot put all education on hold. The situation is particularly challenging for medical students. One might think that the current crisis gives them the opportunity to learn on the front lines. The reality, however, is that there is not enough personal protective equipment (PPE) to protect them. As such, we need to find other ways to maintain their learning trajectory.

A number of medical educators have come up with innovative approaches, and I have thrown my own contribution into the mix. As one who teaches a well-known virtual course that is an introduction to biomedical and health informatics, we are packaging up an offering that we intend to make available as a medical school elective. Because the course is mostly asynchronous, we can scale it up pretty quickly. I don’t just want to throw the materials out there, and still maintain some sort of interaction and connection with learners, but we can offer the course to many students (including those beyond medical students). We plan to launch the first offering to Oregon Health & Science University (OHSU) medical students next week.

I also have an opportunity to advance research related to SARS-CoV-2 in the form of organizing an information retrieval (IR) challenge evaluation. The goal of this retrieval challenge is both to help develop systems capable of identifying relevant information for the current pandemic, but also to scientifically study how retrieval methods can be quickly developed for such situations in the future. The task will follow the "Cranfield" evaluation procedures that are used in the Text Retrieval Conference (TREC) and related challenge evaluations.

This effort is made possibly by work of the Allen Institute for AI and some collaborators who have assembled an open dataset, the COVID-19 Open Research Dataset (CORD-19). This collection of biomedical literature articles currently contains over 40,000 articles and will be updated weekly. Some colleagues and I will be organizing an IR challenge for search engines that find relevant COVID-related articles within this collection. This challenge will provide:
  • A benchmark set of important COVID-related queries (e.g., coronavirus risk factors, COVID-19 ibuprofen)
  • A set of manual judgments for CORD-19 articles on these queries
  • An ongoing leaderboard for comparison of IR systems 
We are even collecting candidate queries in a crowdsourcing manner by asking people to suggest them on Twitter using the hashtag, #COVIDSearch.

The current plan is to run the competition in weekly batches, where that week's snapshot of CORD-19 is used as the corpus and the results of systems participating in that batch are pooled for manual assessment. We will likely use the Kaggle platform to create a “leaderboard” of those whose methods are most effective. The challenge may in the future expand to more detailed tasks such as information-filtering, question-answering, fact-checking, and argument mining.

I make no pretensions that the work I am doing is in any way comparable to front-line healthcare and other essential workers, but I am glad that I can make these contributions that will keep education and research functioning during this tremendous worldwide crisis.

Saturday, March 21, 2020

SARS-CoV-2: The Course Ahead

As the frequency of my postings in this blog has declined in recent years, I have noted several times that the blog started in the frenzied early days of the Health Information Technology for Economic & Clinical Health (HITECH) Act, which was part of the American Recovery and Reinvestment Act (ARRA) and that was instituted in an attempt to blunt the Great Recession of 2008. HITECH was part of ARRA, and of course gave us the big investment that has greatly expanded the adoption of electronic health records (EHRs). A small part of HITECH included investment to build the capacity of the health IT workforce.

Now, of course, we are headed into new economic recessionary times due to SARS-CoV-2, also known as Covid-19 as well as the Novel Coronavirus. Will this be ARRA 2.0?

Before I say anything about reactions to SARS-CoV-2, let me clearly state my sorrow for those most directly affected. Obviously the most sorrow is for those whose lives have been directly impacted by the disease it is causing and also by the disease's impact on their loved ones. There is also sorrow for what is happening to those whose lives are otherwise substantially affected, with threats to their livelihoods or other aspects of their ability to obtain food, shelter, and health care. There is also the impact for those on the front lines, of course in healthcare settings, but also in public safety, grocery stores, and other places of “essential” work. And to a lesser extent the rest of us, obviously minuscule compared to those directly impacted, but with major alterations to our daily lives.

With sorrow does come some opportunity for gratitude. While this is clearly impacting my life, at the end of the day, what I hold most dear - family, friends, and colleagues - are all still there and appreciated for their presence and support. We also owe gratitude for the global Internet, which enables us to keep connected by email, social media, and perhaps most importantly, videoconferencing. A decade ago, the bandwidth and reach of the Internet would not have allowed this level of connection. I am also grateful for my knowledge and experience in online teaching, and how I might put it to work keeping students and faculty connected during these trying times. The latter will likely be a major part of my work effort going forward.

I am certain I will much more to write about in the days ahead. While I had not hoped it would take a crisis to revitalize my blog, it will no doubt do so.

Sunday, February 23, 2020

Adding a New Competency in Clinical Informatics for Medical Education

One of the most widely cited papers I have written in the last decade has been one on competencies in clinical informatics for medical education [1]. For the most part, these 13 competencies have stood the test of time, from knowing how to use the electronic health record and information retrieval systems as well as applying clinical decision support, patient privacy, personal health records, telemedicine, and more. All of these aspects of clinical informatics are essential skills for the 21st century clinicians.

But another area of required competence has come to the fore in recent years, which is the explosion of machine learning and artificial/augmented intelligence in medicine. While the impact of these in real-world clinical practice is still small, the long-term effect is likely to be substantial. Certainly clinicians should be familiar with the myriad of issues related to algorithms and models, including ethical concerns.

In the process of updating our chapter for the forthcoming 2nd edition of the textbook, Health Systems Science [2], my co-author Dr. Jesse Ehrenfeld and I took the opportunity to make this revision to the competencies by adding a 14th one:
14. Apply machine learning applications in clinical care
a. Discuss the applications of artificial/augmented intelligence in clinical settings
b. Describe the limitations and potential biases of data and algorithms

As with the original competencies, we encourage others to improve upon them. But it is also important to add this critical new one to the full set, which are listed below.


1. Hersh, WR, Gorman, PN, et al. (2014). Beyond information retrieval and EHR use: competencies in clinical informatics for medical education. Advances in Medical Education and Practice. 5: 205-212.
2. Skochelak, SE, Hawkins, RE, et al., Eds. (2017). Health Systems Science. New York, NY, Elsevier.

Appendix - Competencies in Clinical Informatics for Medical Education, circa 2020

1.    Find, search, and apply knowledge-based information to patient care and other clinical tasks
a.    Information retrieval/search - choose correct sources for specific task, search using advanced features, apply results
b.    Evaluate information resources (literature, databases, etc.) for their quality, funding sources, biases
c.    Identify tools to assess patient safety (e.g., medication interactions)
d.    Utilize knowledge-based tools to answer clinical questions at the point of care (e.g., text resources, calculators)
e.    Formulate an answerable clinical question
f.    Determine the costs/charges of medications and tests
g.    Identify deviations from normal (labs/x-rays/results) and develop a list of causes of the deviation

2.    Effectively read from, and write to, the electronic health record for patient care and other clinical activities
a.    Graph, display, and trend vital signs and lab values over time
b.    Adopt a uniform method of reviewing a patient record
c.    Create and maintain an accurate problem list
d.    Recognize medical safety issues related to poor chart maintenance
e.    Identify a normal range of results for a specific patient
f.    Access and compare radiographs over time
g.    Identify inaccuracies in the problem list/history/med list/allergies
h.    Create useable notes
i.    Write orders and prescriptions
j.    List common errors with data entry (drop down lists, copy and paste, etc.)

3.    Use and guide implementation of clinical decision support (CDS)
a.    Recognize different types of CDS
b.    Be able to use different types of CDS
c.    Work with clinical and informatics colleagues to guide clinical decision support use in clinical settings

4.    Provide care using population health management approaches
a.    Utilize patient record (data collection and data entry) to assist with disease management
b.    Create reports for populations in different healthcare delivery systems
c.    Use and apply data in accountable care, care coordination, and the primary care medical home settings

5.    Protect patient privacy and security
a.    Use security features of information systems
b.    Adhere to HIPAA privacy and security regulation
c.    Describe and manage ethical issues in privacy and security

6.    Use information technology to improve patient safety
a.    Perform a root-cause analysis to uncover patient safety problems
b.    Familiarity with safety issues
c.    Use resources to solve safety issues

7.    Engage in quality measurement selection and improvement
a.    Recognize the types and limitations of different types of quality measures
b.    Determine the pros and cons of a quality measure, how to measure it, and how to use it to change care

8.    Use health information exchange (HIE) to identify and access patient information across clinical settings
a.    Recognize issues of dispersed patient information across clinical locations
b.    Participate in the use of HIE to improve clinical care

9.    Engage patients to improve their health and care delivery though personal health records and patient portals
a.    Instruct patients in proper use of a personal health record (PHR)
b.    Write an e-message to a patient using a patient portal
c.    Demonstrate appropriate written communication with all members of the healthcare team
d.    Integrate technology into patient education (e.g., decision making tools, diagrams, patient education)
e.    Educate patients to discern quality of online medical resources (Web sites, apps, patient support groups, social media, etc.)
f.    Maintain patient engagement while using an EHR (eye contact, body language, etc.)

10.    Maintain professionalism through use of information technology tools
a.    Describe and manage ethics of media use (cloud storage issues, texting, cell phones, social media professionalism)

11.    Provide clinical care via telemedicine and refer patients as indicated
a.    Be able to function clinically in telemedicine/telehealth environments

12.    Apply personalized/precision medicine
a.    Recognize growing role of genomics and personalized medicine in care
b.    Identify resources enabling access to actionable information related to precision medicine

13.    Participate in practice-based clinical and translational research
a.    Use EHR alerts and other tools to identify patients and populations eligible for participation in clinical trials
b.    Participate in practice-based research to advance medical knowledge

14.    Apply machine learning applications in clinical care
a.    Discuss the applications of artificial/augmented intelligence in clinical settings
b.    Describe the limitations and potential biases of data and algorithms

Tuesday, December 31, 2019

Annual Reflections at the End of 2019

It has been customary for me to post an annual reflection at the of end of the year for this blog. The world of blogging, and my own blogging, have certainly changed over the years. This is probably due in part to social media reaching a level of maturity or, in the parlance of the Gartner Hype Cycle, the plateau of productivity. When I started this blog, blogging was relatively new. In parallel was the excitement of the HITECH Act, which certainly transformed the world of biomedical and health informatics.

But I still enjoy blogging, and appreciate the forum it provides me to speak to things of interest in informatics. This past year included one milestone, which was the blog passing the one-decade old milestone from the first posting on March 12, 2009. But my frequency of posting has gone down over the years. This is only my 15th post for 2019, down from a peak of 44 posts in 2013.

Still, 2019 was another excellent year, and I remain gratified to have a career I find rewarding. One number that has achieved relevance for me as the year ends is 2600. This number represents two things. One is the number of people who have completed the OHSU offering of the AMIA 10x10 course. The other is the number of people who follow me on Twitter. My gradual embrace of Twitter is noteworthy because I posted several years ago that I did not find much use for it. My social media use has generally settled into using Twitter for professional purposes and Facebook for mostly personal uses, although many of my Facebook friends are also professional colleagues, so there is overlap with professional activity there.

I also have much gratitude for all the other wonderful things in my life, including my family, my friends and colleagues, and my health. And as 2019 draws to a close, we now enter a new decade, the twenties. I could never imagined many of the things that happened, good and bad, in the 2010s, so it will be interesting to see how the twenties develop.

Tuesday, December 3, 2019

Eligibility for the Clinical Informatics Subspecialty - 2019 Update

I have been posting periodic updates of the eligibility for the physician clinical informatics subspecialty since the early days of the subspecialty, motivated by regular receipt of emails from physicians asking me questions about their individual eligibility. Rather than reply from the beginning each time, I replied with a link to the latest posting on eligibility in this blog and then instruct them to read the posting and write back to me with any questions specific to their situation. I also pointed out that my recommendations were just my interpretation of the rules, which were officially set by the American Board of Preventive Medicine (ABPM) for physicians of all specialties other than pathology, which were set by the American Board of Pathology (ABPath).

The very first posting was in 2013, which was then superseded by several minor updates. The overall eligibility rules have not changed much: As a subspecialty of all medical specialties, one must have a primary board certification in one of the 23 physician specialties that are recognized by the American Board of Medical Specialties (ABMS). Unfortunately, this excludes physicians who never achieved primary specialty certification or whose primary certification had lapsed. (Most board certifications now must be renewed every 10 years, although some of us trained in the era when boards such as the American Board of Internal Medicine [ABIM] granted lifetime certification.)

Perhaps the most significant update to my periodic postings came after 2016, when ABPM and ABPath extended the "grandfathering period" for an additional five years from 2017 to 2022. As with most new specialties and subspecialties, the clinical informatics subspecialty has a grandfathering period allowing one to become eligible to sit for the board exam and get certified through a certain level of practice or educational ("non-traditional" fellowship or master's degree) attainment.

With less than three years to go before the end of the grandfathering period, the window of opportunity for the practice or educational pathways is closing, especially for those who do not yet have any formal work or training in the field. It is unlikely that ABPM and ABPath will extend the grandfathering period beyond 2022. After that time, the only way to achieve board eligibility will be through a fellowship accredited by the Accreditation Council for Graduate Medical Education (ACGME).

For physicians who might be able to achieve eligibility via the Practice Pathway, my same advice from 2013 still holds: It will be more difficult, especially for mid-career physicians, to achieve board eligibility in or after 2023, when the only pathway to board certification will be to complete an ACGME-accredited fellowship. One other option for physicians will be the Advanced Health Informatics Certification being developed by AMIA. This will be a certification open to all who work in informatics but will be a pathway to certification for physicians who are not eligible for the ABMS clinical informatics subspecialty.

One fortunate happening since 2013 has been the marked improvement of the ABPM Web site, including its page describing eligibility for the clinical informatics subspecialty. (It was always somewhat ironic that the original Web site for the clinical informatics subspecialty was so poorly designed.) Another improvement of this page is the simplification of the explanation for becoming board-eligible, with all "grandfathering" now folded into the Practice Pathway. The site notes two options are available in the Practice Pathway (from which I quote):
  • Time in Practice: Three years of practice in Clinical Informatics is required. Practice time must be at least 25% of a Full-Time Equivalent (FTE) to be considered. Practice time need not be continuous, however, all practice time must have occurred in the five-year period preceding June 30 of the application year. Practice must consist of broad-based professional activity with significant Clinical Informatics responsibility. Fellowship activity that is less than 24 months in duration or non-ACGME accredited may be applied toward the practice activity requirement. The actual training must be described for any fellowship activity. Documentation of Clinical Informatics research and teaching activities may also be submitted for review.
  • Masters or PhD in Biomedical Informatics: Credit for completion of a 24 month Masters or PhD program in Biomedical Informatics, Health Sciences Informatics, Clinical Informatics, or a related subject from a university/college in the US and Canada, deemed acceptable by the Board (e.g. NLM university-based Biomedical Informatics Training) may be substituted for the Time in Practice option above.
So what are the options for clinical informatics board subspecialty eligibility in late 2019? Since the 2022 exam is now less than three years away, the Time in Practice pathway is closed unless one has already been in a qualifying position since mid-2019 or earlier. Fortunately, the Masters pathway is still open, and our online Masters program at Oregon Health & Science University (OHSU) fits the criteria of the second bullet above, namely our being part of a program that is a National Library of Medicine (NLM) university-based training program. Indeed, we have a number of physicians who have enrolled in the program with the intent of graduating by June 2022 and becoming board-eligible for the 2022 exam. Several of them have formed a cohort to work together toward that goal.

Saturday, November 16, 2019

Informatics Has Arrived ... in the World of Fiction

You know that your scientific field has arrived when it shows up in fiction. Informatics has now reached that point, as the field is featured in two new novels.

One is written by a recently retired informatician, Perry Miller, MD, PhD of Yale University. Dr. Miller has transitioned to becoming a novelist, authoring the book, Lethal Injection (Koehler Books, Virginia Beach, VA). In this murder mystery based in a hospital, IT systems play a significant role in the story, and one of the central characters has a master’s degree in informatics from OHSU. I won’t give away the rest of the story, but can say it was an enjoyable book to read.

In the other book, the characters from House of God, a famous novel from when I was in medical school in the 1980s, are reunited at Man's 4th Best Hospital, which is also the name of the book (Penguin Publishing Group). The characters are brought back together in an effort to defeat HEAL, the “Healthy Electronic Assistance Link” electronic health record foisted upon this flailing health system in an effort to improve its bottom line … and maybe also improve patient care. HEAL was developed by “electrical engineering grads, isolated out in Cheese Country, Wisconsin.” Anyone who knows anything about informatics knows the vendor for which those HEAL developers work.

Both books also deal with another problem in medicine, which is the consolidation of healthcare systems and resulting emphasis on the bottom line, sometimes to the detriment of care and well-being of clinicians. But both books are nice reads, and I won't say more to spoil their stories, other than to note it is interesting to see informatics come of age in fiction.

Monday, November 4, 2019

Moving FHIR from Aspirational to Operational

After years of giving lip service to standards, the health information/informatics community is now taking interoperability very seriously. The reason is obvious: we have spent a decade ramping up adoption of electronic health records (EHRs) in the US and elsewhere, and we have learned in hindsight that in the hurry to get systems implemented as quickly and easily as possible, inadequate attention was paid to data standards and interoperability. As a result, we have EHRs across the health system that do not easily talk to each other. The has compromised the hopes we have had for simple health information exchange (HIE) [1], not to mention the myriad re-uses of EHR data for research and quality assurance data [2].

The adoption of standards was also hindered by problems with the existing HL7 messaging standards, from the venerable but limited Version 2 to the hopelessly complex Version 3. Then, being the right solution at the right time, along came the Fast Healthcare Interoperability Resources (FHIR) standard [3,4].

Demonstrating a strong need for improved interoperability, the uptake across healthcare has been swift. FHIR received an early boost by being synergistic with the SMART app framework [5]. Another early enthusiast was ONC, which baked FHIR into the new rules mandated from the 21st Century Cures Act. (I view the health IT aspects of this legislation as a way to clean up the insufficient attention to interoperability in the original HITECH Act).

Other communities have jumped on the bandwagon as well. The new push to make quality measures easier to extract from the EHR, as opposed to requiring manual extraction from chart reviews, in the electronic Clinical Quality Measures (eCQMs) project.  The research community has joined as well, with the National Institutes of Health (NIH) calling for its use. Other research groups have started to develop tools, such as in natural language processing (NLP) pipelines [6] and the Clinical Data to Health (CD2H) data coordinating center for the Clinical & Translational Science Award (CTSA) program.

Other uptake of FHIR includes its use in clinical decision support tools. The value-based care community has jumped on board as well in the Da Vinci Project that focuses on managing and sharing clinical and administrative data.  There are even applications in the education arena, with the description of a tool recently developed to use SMART on FHIR in a case-based learning situation [7].

Despite all the excitement and achievement, the operational use of FHIR remains modest. This is not to argue that it will not achieve success across the healthcare system, but at the present time its use in real-world situations is small. But the enthusiasm shows that the needs it is intended to address are real, and that it has the potential to provide effective solutions for interopreability problems.

One example of a great start but need for more development involves the Apple Health app on my iPhone. I love to pull out my phone and show anyone who wants to see that I can download a good portion of my medical record from my institution's Epic EHR system to the Apple Health app, even the function that lets me show the FHIR resources in raw XML. But the reality is that at the present time, about all I can do with this app is show the presence of my data to people.

There are still some unanswered questions about making FHIR operational, such as:
  1. How will the FHIR approach scale up to large and diverse data types from the EHR and beyond?
  2. Will we be able to transform the unstructured data in notes and other parts of the record into the detailed structured form of FHIR?
  3. How will people manage the data that leaves the healthcare system, especially consumers who may not be savvy about their medical data they now possess on their phones and other devices?
We therefore need to think at the present time of FHIR as more aspirational than operational. That said, we need to leverage the widespread enthusiasm across the healthcare system to build on the current foundation to carry out the real work that must be done to reach the goals that everyone has for standards-based, interoperable data.


1. Dixon B. Health Information Exchange - Navigating and Managing a Network of Health Information Systems. Amsterdam, Netherlands: Elsevier; 2016.
2. Meystre S, Lovis C, Bürkle T, Tognola G, Budrionis A, Lehmann C. Clinical data reuse or secondary use: current status and potential future progress. In: Holmes J, Soualmia L, Séroussi B, editors. Yearbook of Medical Informatics. 262017. p. 38-52.
3. Benson T, Grieve G. Principles of Health Interoperability - SNOMED CT, HL7 and FHIR, Third Edition. London, England: Springer; 2016.
4. Braunstein M. Health Informatics on FHIR: How HL7's New API is Transforming Healthcare. New York, NY: Springer; 2018.
5. Mandel J, Kreda D, Mandl K, Kohane I, Ramoni R. SMART on FHIR: a standards-based, interoperable apps platform for electronic health records. Journal of the American Medical Informatics Association. 2016;23:899-908.
6. Hong N, Wen A, Shen F, Sohn S, Wang C, Liu H, et al. Developing a scalable FHIR-based clinical data normalization pipeline for standardizing and integrating unstructured and structured electronic health record data JAMIA Open. 2019: Epub ahead of print.
7. Braunstein M, Oancea I, Barry B, Darlington S, Steel J, Hansen D, et al. The development and electronic delivery of case-based learning using a Fast Healthcare Interoperability resource system. JAMIA Open. 2019: Epub ahead of print. 

Wednesday, September 18, 2019

Closing the Loops on Data Science and Informatics

One of the most highly viewed posts of this blog is a 2015 posting, What is the Difference (If Any) Between Informatics and Data Science. One critique I have had of data science is the focus of most work on only showing prediction and not implementing prescription. In other words, how do we take the predictive output in an ever-increasing number of areas of biomedicine and turn it into programs that actually improve outcomes, whether better patient care, improved healthcare delivery, or more effective research? Some recent publications bring this issue to light and show that we have some loops to close before we attain the value of data science in biomedicine.

A couple of recent perspective pieces bring this closure into light. One is from colleagues Philip Payne, Elmer Bernstam, and Justin Starren [1]. In a Perspective last year in JAMIA Open, they put forth a model that delineates the loop that must be closed, from the development of data science (and informatics) models and systems to the real-world informatics that most who work in the field are familiar with of implementing and evaluating systems with real users and organizations. A more recent paper from Lenert et al. notes that as predictive models are put into place and impact outcomes, they will necessarily impact those models, which will need to be adjusted to the new reality of their use [2].

One aspect of this first loop to be closed is how we study data science and machine learning interventions in actual clinical practice. A pair of recently published papers demonstrate how models and systems can be built and validated, and then assessed in the clinical real world. A first paper by Barton et al. develops and evaluates a model for predicting sepsis from patient vital designs [3]. Sepsis is a medical problem of continued significance while vital sign data is readily available. A subsequent paper by Shimabukuro et al. implements a randomized controlled trial in two medical intensive care units, finding a decrease in length of stay in the units from 13.0 to 10.3 days and a 12.4% reduction in in-hospital mortality [4].

Another recent study assessed the application of machine learning to detecting colonic polyps during colonoscopy [5]. While the machine learning system worked effectively, it was mostly effective at recognizing polyps that were unlikely to progress to cancer quickly, such as small adenomas and hyperplastic polyps. Nonetheless, recognizing such polyps improves the overall quality of colonoscopy exam.

A second loop that will need to be closed to achieve the vision of widespread generalized application of data science will be the generation of standardized EHR data for use across the healthcare system. A group of colleagues and I wrote about this in 2013 [6], as have many others, but some recent work documents aspects of this problem are still not solved. Two recent analyses show variations in how physicians [7] and healthcare organizations [8] document patient care, which may lead to variation in data that is not due to underlying differences in patients.

The need to close these loops show we are still in the early days of machine learning and predictive algorithms. While their impact in medicine will likely be enormous in the long run, there is still much work that will need to be done to optimize their data and how they are most effectively used.


1. Payne P, Bernstam E, Starren J. Biomedical informatics meets data science: current state and future directions for interaction. JAMIA Open. 2018;1:136-41.
2. Lenert M, Matheny M, Walsh C. Prognostic models will be victims of their own success, unless. . . Journal of the American Medical Informatics Association. 2019; Epub ahead of print.
3. Barton C, Chettipally U, Zhou Y, Jiangce Z, Lynn-Palevsky A, Le S, et al. Evaluation of a machine learning algorithm for up to 48-hour advance prediction of sepsis using six vital signs. Computers in Biology and Medicine. 2019;109:79-84.
4. Shimabukuro D, Barton C, Feldman M, Mataraso S, Das R. Effect of a machine learning-based severe sepsis prediction algorithm on patient survival and hospital length of stay: a randomised clinical trial. BMJ Open Respiratory Research. 2019;4(1):e000234.
5. Wang P, Berzin T, Brown J, Bharadwa S, Becq A, Xiao X, et al. Real-time automatic detection system increases colonoscopic polyp and adenoma detection rates: a prospective randomised controlled study. Gut. 2019; Epub ahead of print.
6. Hersh W, Weiner M, Embi P, Logan J, Payne P, Bernstam E, et al. Caveats for the use of operational electronic health record data in comparative effectiveness research. Medical Care. 2013;51(Suppl 3):S30-S7.
7. Cohen G, Friedman C, Ryan A, Richardson C, Adler-Milstein J. Variation in physicians' electronic health record documentation and potential patient harm from that variation. Journal of General Internal Medicine. 2019; Epub ahead of print.
8. Glynn E, Hoffman M. Heterogeneity introduced by EHR system implementation in a de-identified data resource from 100 non-affiliated organizations. JAMIA Open. 2019; Epub ahead of print.

Wednesday, August 21, 2019

An Information Retrieval Researcher’s Peer Review of Recent Studies of Search Engine Influence on Voting Behavior

A good part of my informatics research work over three decades has focused on the evaluation of search, also called information retrieval or IR. I have been amazed as the reach of search systems has become a mainstream part of our society, especially given that when I started, IR systems were only used by those who had computers with accounts from companies offering subscription search services.

Now, however, searching is ubiquitous. Indeed, it is almost impossible not to search, as it is offered in the address bar of most Web browsers. In addition, the name of one famous search engine, Google, has become a verb that synonymous with searching, i.e., Googling. Few of us can imagine a world without information on almost any topic being available nearly instantaneously.

It was therefore of interest this week when the President of the United States latched on to some research purporting that manipulation of Google was responsible for shifting three or more million votes to Hillary Clinton, which happens to be the amount of popular votes that she received over Donald Trump in the 2016 election (despite his narrow victory in the Electoral College).

This research has been put forth by Robert Epstein, PhD, who claims to be a liberal Democrat, as if that somehow indicates his analysis is not biased. Of course, one’s political views should not have any influence over the outcomes of their research.

Let’s look at Epstein’s multifaceted claims and the evidence supporting them from the standpoint of an IR researcher. First is the “finding” that Google manipulated search results to retrieve information “biased” toward Clinton. And second is that the retrieval of this information resulted in shifting of votes from Trump to Clinton.

The finding of manipulated search results comes from the paper posted as a PDF to Epstein’s Web site. As such, it is not peer-reviewed. The paper claims to show that in the run-up to the 2016 election and afterwards, 95 individuals, 21 of them of whom designated themselves as “undecided,” had their Google searches tracked and sent to a crowdsourcing site, Mechanical Turk, for rating as to whether they were biased toward Clinton or Trump. They eliminated searches from people who had Gmail accounts due to an unsubstantiated assertion that Google provided such users different results (which the company denies).

If I were sent this paper for peer review by an IR journal, I would ask the following: How did the researchers choose the individuals for the study? What evidence supports excluding those who had Gmail accounts? Who were the people on Mechanical Turk who did the ratings for the study? How were they instructed by the researchers to determine “bias?” I would certainly demand answers to questions like these before I would recommend acceptance for publication. The Methods section of the paper would need to be substantially expanded.

Let’s say, however, that the authors came back with acceptable answers to my questions, and the study were published. What about the second claim that this bias could lead to “manipulating” anywhere from 2.6-10.4 million votes in Clinton’s favor? The evidence for this comes from a paper that was published in a peer-reviewed journal, a prestigious one at that, the Proceedings of the National Academy of Sciences (PNAS). That study, published in 2015, looked at five randomized trials assessing the “search engine manipulation effect” (SEME).

These studies may be credible, but it is dubious whether they can be used to claim biased search results may have impacted voting in the US 2016 election. The first three experiments in the PNAS paper recruited individuals in the San Diego, CA area to rate who they might vote for the two candidates in the Australian Prime Minister election (chosen because most in San Diego would be unlikely to have prior knowledge). A fourth experiment replicated the first three with a national audience of individuals recruited from Mechanical Turk, while a fifth experiment recruited undecided voters to assess information about candidates in a local election in India. There should be no question that any kind of exposure to information can influence one’s decision about voting, although it would be questionable whether these sorts of results could be applied to a national US election where these same people would be bombarded by articles, reports, advertising, and other sorts of information, perhaps even Fake News promulgated by foreign entities on Facebook or Twitter.

Epstein fused the results of this research together to claim that biased search results moved several million votes in the direction of Clinton in the 2016 election. He took this conclusion to a receptive audience of Republicans in the US Senate. The outcome was predictable, with no skepticism whatsoever. And then came the crowning glory of it all, a Presidential tweet.

The mainstream fact checkers had a field day with these claims. Clearly one incompletely reported and probably highly flawed study, fused with another one showing that in some instances, search results can influence voting behavior, is hardly evidence that alleged bias by Google moved votes to Clinton in 2016. Here are some of their assessments:
The results of all this remind of the famous joke by comedian Stephen Colbert, who once noted, reality has a liberal bias. I do believe it is important and I strive to keep political biases out of our research. Even in my teaching, I aim to present opposing points of view, although not aiming to give equivalence to all points of view. But research like this needs to be called out for its thinly veiled political goals, and I suppose on that front, its “results" can be called successful.

Sunday, August 4, 2019

Living the Asynchronous Life

I have always been highly productive in my academic career, finding time to teach, carry out research, write, manage my department, and mentor faculty and students. I believe that one of the reasons for my productivity is my leading a relatively asynchronous life.

What do I mean by asynchronous? In essence, most of my work is done by myself, wherever and whenever. And if I hit a block (or get tried of something), I can leave one task and move to another.

A good part of the asynchronous nature of my work comes from my teaching, which these days is mostly online. Most of my courses involve recorded (though frequently updated) lectures delivered online, followed by online discussion through the Oregon Health & Science University (OHSU) learning management system. I also interact with students by email and even sometimes by phone.

This mostly asynchronous work is also amenable to my lifestyle. Being a (very early) morning person, I can get most of my important and creative work done early in the day. A typical day for me involves 2-4 hours of work, exercise, and arriving at the office between 9-10 am. (Leading some to believe I just roll into the office late, when I have in fact have already been working for several hours.)

Another advantage of my mostly asynchronous work is that it allows me to pursue other aspects of my job that I enjoy, such as travel. I have often noted that I get some of my best work done in hotel rooms during travels. In almost every corner of the world, I can get access to the Internet, all my internal OHSU resources, and OHSU's learning management system. Having 3-4 hours uninterrupted in a hotel room can result in completing a day’s worth of work. And while not all of my work is amendable to being done on airplanes, I can choose work that is when flying. (It does not hurt to have status that gives me more roomy seating (and even sometimes upgrades).

There are some downsides to the asynchronous life. One is that the capacity for switching across tasks is not infinite. As with multitasking computers, too many tasks, and switching between them too frequently, can lead to so-called deadlock.

Another downside is that it can sometimes seem that work never ends. Even when a project is completed, there are others still demanding attention.

Nonetheless, given a combination of my satisfaction of creating academic papers, talks, and hire, plus my enjoyment of getting to see the world, I will always enjoy living the asynchronous life.

Tuesday, July 16, 2019

The Next Chapter in Continuing Education for Informatics

This week, the Department of Medical Informatics and Clinical Epidemiology (DMICE) of Oregon Health & Science University (OHSU) launched a new annual continuing education (CE) activity in clinical informatics. With the first offering of the OHSU Annual Update in Clinical Informatics, a selection of important topics will be covered to provide an update for all clinical informatics professionals. For physicians, the course will provide continuing medical education (CME) credit. For physicians certified in the clinical informatics subspecialty, the course will provide MOC-II/LLSA credits.

The field of clinical informatics (and closely related health informatics) is a growing profession that plays an important role in healthcare and other health-related areas [1]. Informatics professionals insure that data and information are used most effectively to improve healthcare, public health, individual health, and research. The certification initially of physicians [2] and soon others in the field [3] requires that all informatics professionals maintain and expand their knowledge and skills.

This course builds off the extensive informatics education offerings of DMICE, from our biomedical informatics graduate program that has awarded 831 degrees and certificates over more than 20 years to our other innovative activities such as the AMIA 10×10 (“ten by ten”) program, the development of online learning in informatics, and launching one of the first clinical informatics subspecialty fellowships for physicians [4].

The learning activity consists of 7 modules that are estimated to take a total of 8 hours to complete. The activity is completely online, and consists of lectures and self-assessment quizzes. The topics for the 2019 annual update were selected by DMICE faculty. Topics for future annual updates will be chosen with input from those who completed previous annual update courses.

After taking this learning activity, clinical informatics professionals will be able to (1) be aware of current advances in clinical informatics. (2) apply these advances to their professional practice, and (3) meet required competencies that are related to the domain of clinical informatics in the practice of their profession.

The activity will consist of a number of talks given by DMICE faculty that will focus on recent developments in the field. The activity will be hosted on OHSU’s Sakai learning management system as enduring learning material. Once learners enroll in the activity, they will have access to Sakai and be able to complete the activity and evaluations at their own pace. Each talk will be accompanied by a post-test (multiple choice), and learners will also need to complete a course evaluation at the end of their learning. The 2019 course activities must be completed by June 30, 2020.

The topics covered in this year’s offering of the course include:
  • Operational Clinical Informatics
  • Organizational Behavior
  • Data Science and Machine Learning
  • Clinical Research Informatics
  • Informatics Education
  • Nursing Informatics
Details of this online CE experience are available at:

This is not the only continuing education activity in clinical informatics that will be offered by OHSU. In the coming year, we will also offer for CME and ABPM MOC-II credit our monthly clinical informatics journal clubs and grand rounds.


1. Fridsma, D. (2019). Strengthening our profession by defining clinical and health informatics practice. Journal of the American Medical Informatics Association, Epub ahead of print.
2. Detmer, D., & Shortliffe, E. (2014). Clinical informatics: prospects for a new medical subspecialty. Journal of the American Medical Association, 311, 2067-2068.
3. Gadd, C., Williamson, J., Steen, E., & Fridsma, D. (2016). Creating advanced health informatics certification. Journal of the American Medical Informatics Association, 23, 848-850.
4. Longhurst, C., Pageler, N., Palma, J., Finnell, J., Levy, B., Yackel, T., . . . Hersh, W. (2016). Early experiences of accredited clinical informatics fellowships. Journal of the American Medical Informatics Association, 23, 829-834.

Monday, July 8, 2019

Kudos for the Informatics Professor - Winter/Spring 2019 Update

I have had a busy but productive early 2019, with invited talks, publications, and other happenings.

I gave a few invited talks:
The latter was a real honor, as it took place at my medical school alma mater, University of Illinois Chicago, and it was fun to see both informatics colleagues as well as some former classmates who attended the lecture.

I also had the opportunity to attend the inaugural induction of Fellows of the American Medical Informatics Association (FAMIA) at the AMIA Clinical Informatics Conference in Atlanta, GA on May 1, 2019. The initial group of FAMIA included 15 alumni and faculty of the OHSU Biomedical Informatics Graduate Program, comprising over 11% of the inaugural fellows. Below is a picture of OHSU alumni and myself who attended the induction ceremony.

I continue to serve on several scientific advisory boards:
  • Pan African Bioinformatics Network for H3Africa (H3ABionet), which provides bioinformatics support for the Human Heredity and Health in Africa Project (H3Africa). I will be attending my second meeting of the board in Cape Town, South Africa in July.
  • RCMI Multidisciplinary And Translational Research Infrastructure EXpansion (RMATRIX), a translational research center of the John A. Burns School of Medicine of the University of Hawaii. The grant funding of this is ending, so this board will also be ending.
  • ECRI Guidelines Trust (EGT) Technical Advisory Panel (TAP), a publicly available web-based repository of objective, evidence-based clinical practice guideline content that succeeds the Agency for Healthcare Quality & Research (AHRQ) National Guidelines Clearinghouse. (I also served on the Technical Expert Panel of National Practice Guidelines Clearinghouse when it was developed and launched by AHRQ from 1998-2002.)
I am also co-author on a couple papers that were published:
I also had the pleasure, as I do every June, of seeing a new group of graduates from the OHSU Biomedical Informatics Graduate Program. This year saw 45 new alumni of the program honored. With these new graduates, the program has now awarded a total of 831 degrees and certificates dating back 22 years to the first graduates in 1998 (who started when the program launched in 1996). As some have completed more than one program degree or certificate (e.g., the Graduate Certificate and Master’s or the Master’s and PhD; one person has done all three!), the program has a total of 746 alumni.

Finally, AMIA has produced and posted videos for several of the courses in the 10x10 program. A video of myself describing the OHSU course has been posted on the main page for the program on the AMIA Web site.

Monday, June 24, 2019

Introducing Informatics.Health

About a year ago, the Internet Assigned Numbers Authority (IANA) launched a new set of top-level Internet domains. One of these was .health. When to my surprise, the domain name was still available, I immediately grabbed it. I am now pleased to launch my first use of the domain name, which is a re-direct to my well-known site, What is Biomedical and Health Informatics?

In honor of this launch, I have completely updated the "What is...?" site, which I use to both provide an overview of the field to those interested and also demonstrate the online learning technologies that we use in our Biomedical Informatics Graduate Program at OHSU.

The main part of the site consists of the following lecture segments (time in parentheses):
  • What is Biomedical and Health Informatics (1)? (23:30)
  • What is Biomedical and Health Informatics (2)? (17:41)
  • A Short History of Biomedical and Health Informatics (21:33)
  • Resources for Field: Organizations, Information, Education (24:37)
  • Examples of the Electronic Health Record (EHR) (24:08)
  • Data Science and Machine Learning (1) (14:26)
  • Data Science and Machine Learning (2) (20:09)
  • Information Retrieval (Search) (24:05)
  • Information Retrieval Content (29:26)
The site also contains links to books, articles, organizations, and educational Web site.

Over time, I will probably move the site to a new server, and eventually I may develop different content for it. However, I will always want the site to be an overview of the biomedical and health informatics field. 

Saturday, June 22, 2019

Recovering from a Computer Demise, 21st Century Edition

My professional and personal lives have involved computer use for many decades. As time has gone on, the proportion of my life tied to computing devices has increased. Whereas in the early days I mainly saved programs and simple documents, my life is now intertwined with computers, smartphones, and tablets; covering virtually everything I do professionally and also a large amount of personal activities, from pictures to music to documents and more.

Although computers are more reliable than ever, the impact of a failure is more catastrophic in the present when so much of one's life is tied up in them. The recent sudden demise of my MacBook Pro reminded me of this.

Back in the early days, as well as now, the most critical problem where is a computer failure is loss of one's data. Because data loss was more common in those early days, I have always backed up my data frequently. In modern times, the easiest way to do this as a Mac user is through the use of Time Machine, which is built into the operating system. I keep two hard disks for this purpose, one to take with me whenever I travel and another to stay home in case everything is lost or stolen on a trip. Then and now, I have always backed up my data about once a day. This habit fortunately minimized the impact of my recent computer demise.

I have actually experienced only a handful of computer failures in my decades of using them. But a few weeks ago, while flying home from Singapore, fortunately on the last leg of the trip from San Francisco to Portland, my MacBook Pro just died. No amount of trying to reset the System Management Controller (SMC) or anything else helped. The battery was far from dead, so plugging in the computer did not help either. The death was verified by the OHSU IT department after I landed and brought it in to them.

Fortunately my department had a recently re-imaged MacBook Pro for me to use immediately. But most fortunately, I had backed up my now-dead machine about 24 hours earlier.

When I get a new Mac, I generally prefer not to restore the entire computer image in Time Machine. Even though it would be faster, I know of those who have had problems with this approach, and I prefer to re-build my machine by re-installing the data and then the individual apps. I also like the opportunity to do some "housecleaning" to get rid of applications I am not actively using and mostly clutter machine.

(I also have a systematic method for backing up all data I want to maintain in the long run. I have been doing this as well since the late 1980s and have an archive of essentially my entire career, even though some files from those early days no longer have applications that can open them. Microsoft Office, for example, will not open those Word/Excel/Powerpoint-format files from those times, but the files can be opened by a text editor.)

With the new MacBook Pro, I was quickly able to restore my data from my Time Machine disk, which gave me all of my data from about 24 hours prior. This meant I would be losing all work I had done in the last 24 hours since that backup, which was not insubstantial, since I had been working during my last hours in Singapore and then my long trans-Pacific flight. I was able to retrieve a few things I had done within those 24 hours, for example documents I composed and sent by email, which were in the outbox of my mail client. (Doesn’t hurt to have had wifi on my long flight!)

It generally takes me a couple weeks to get a new computer fully restored from a prior one, and this case was no different. The MacBook Pro was sent to Apple for repairs and they had to replace the entire innards, so there was no way to recover anything from the old computer. But due to careful backing up and other processes, my computer demise was fortunately not too painful, and I re-learned the lesson of regularly backing up one's work.

I know there are also in modern times some processes that eliminate the need for users to actively back up their work, such as to a cloud-based location. But even this would be imperfect for me, since my Internet access is not yet completely ubiquitous (such as when the airplane wifi does not work or a local connection is too expensive or otherwise not available). So I imagine that my habit of regularly backing up my data will be a good one to keep for some time to come.

Thursday, April 11, 2019

Beyond Images and Waves: How Will Deep Learning Benefit Health or Healthcare?

Scarcely a week goes by without another study published of a deep learning algorithm that achieves accuracy comparable to or sometimes better than human experts. The vast majority of these studies focus on some application to diagnostic imaging or waveform interpretation. There is no question that the practice of medicine will be highly impacted by these systems, especially for practitioners in fields that directly use them, such as radiologists, pathologists, dermatologists, and so forth.

What about deep learning applied to other areas of medicine? One group of studies has applied deep learning to retrospective electronic health record (EHR) data. A number of studies have shown impressive abilities to use EHR data to predict or diagnose:
  • Several dozen diseases [1]
  • Length of stay, mortality, readmission, and diagnosis at two large medical centers [2]
  • Prognosis in palliative care [3]
  • 30-day readmission in heart failure [4]
  • Patient mortality from coronary artery disease more accurately than traditional cardiovascular risk models [5]
  • Early risk of chronic kidney disease in patients with diabetes [6]
  • Many pediatric diagnoses at a major referral center [7]
  • Clinical outcomes in rheumatoid arthritis [8]
One can see obvious use cases for these types of systems, such as being able to intervene when patients might fare poorly when hospitalized or are at risk for readmission or more serious outcomes after they are discharged. But making a faster or more accurate diagnosis from an imaging or waveform study is a different matter than trying to determine the best use of an algorithm that tells a clinical team that a patient may be heading toward a bad outcome. When do we apply the results of the system? Once we intervene, does this change the nature of future recommendations? These are fascinating research questions, but also big unknowns in terms of how to apply such data in the clinical setting.

I wrote back in 2014, and more recently in 2017, that these systems must go beyond their ability to predict and actually be used in the context of prescriptive systems that results in better outcomes for the patient and/or the healthcare system. I hope to see studies going forward where these sorts of systems become part of the patient care team, and lead to demonstration of Friedman’s Fundamental Theorem of Informatics [9], which is that humans aided by machines do better than humans or machines alone. A nice roadmap for such studies was recently published that notes the need for studies that include meaningful endpoints, appropriate benchmarks, transportable to other settings and systems, and including legal and ethical monitoring [10].

This sentiment is echoed in the excellent new book by Dr. Eric Topol, Deep Medicine [11]. Dr. Topol raises the notion of deep learning benefitting patient care in even additional ways. One could solve a problem that has vexed healthcare with the widespread adoption of EHRs, which is the introduction of a third entity into the patient-physician encounter, namely the computer. Clinicians now unfortunately spend too much time “feeding the beast,” especially when that beast distracts from the patient and has additional compliance and billing burdens, contributing heavily to our epidemic of burnout in clinicians [12].

Perhaps one area where deep learning might help reduce the clinician burden is in data entry. An intriguing new study was recently published by one of the same authors who contributed some of the EHR suites above, automatically charting symptoms from patient-physician conversations [13]. One also wonders whether the level of a patient visit for billing purposes, currently determined by the presence of various elements document in the medical record, could be replaced by other data easy to collect in the modern medical office, such as time spent with the patient, time with other aspects of care, and other requirements of care. While we may never be able to achieve a “computer-free” patient examination room, we will hopefully find ways to reduce its impact and burden.

Dr. Topol suggests a second major area of benefit for deep learning, which is monitoring patients in much more depth than our current approach to providing episodic healthcare. Of course, this must be done in ways that provide actionable information presented in ways that do not further overburden clinicians. But it is not beyond the pale to envision his view of algorithms the coalescence of -omics, personal sensors, physiological measurements, healthcare, and public health data coming together to give diagnostic, therapeutic, and prognostic advice. There are still many issues around cost, ethics, practicality, and so forth, so the benefits will need to outweigh the risks.

Thus, while we will likely see machine assistance in diagnosis from images and wave forms in the near future, the impact of what deep learning may do with patient data in the EHR and with the patient will likely take longer. I see exciting opportunities for research and development focus on how to prospectively determine how such systems fit into the workflow of patient interaction within and outside the healthcare system. A nice overview

1. Miotto, R, Li, L, et al. (2016). Deep Patient: an unsupervised representation to predict the future of patients from the electronic health records. Scientific Reports. 2016: 26094.
2. Rajkomar, A, Oren, E, et al. (2018). Scalable and accurate deep learning for electronic health records. npj Digital Medicine. 1: 18.
3. Avati, A, Jung, K, et al. (2018). Improving palliative care with deep learning. BMC Medical Informatics & Decision Making. 18: 122.
4. Golas, SB, Shibahara, T, et al. (2018). A machine learning model to predict the risk of 30-day readmissions in patients with heart failure: a retrospective analysis of electronic medical records data. BMC Medical Informatics & Decision Making. 18: 44.
5. Steele, AJ, Denaxas, SC, et al. (2018). Machine learning models in electronic health records can outperform conventional survival models for predicting patient mortality in coronary artery disease. PLoS ONE. 13(8): e0202344.
6. Ravizza, S, Huschto, T, et al. (2019). Predicting the early risk of chronic kidney disease in patients with diabetes using real-world data. Nature Medicine. 25: 57-59.
7. Liang, H, Tsui, BY, et al. (2019). Evaluation and accurate diagnoses of pediatric diseases using artificial intelligence. Nature Medicine. 25: 433-438.
8. Norgeot, B, Glicksberg, BS, et al. (2018). Assessment of a deep learning model based on electronic health record data to forecast clinical outcomes in patients with rheumatoid arthritis. JAMA Network Open. 2(3): e190606.
9. Friedman, CP (2009). A 'fundamental theorem' of biomedical informatics. Journal of the American Medical Informatics Association. 16: 169-170.
10. Parikh, RB, Obermeyer, Z, et al. (2019). Regulation of predictive analytics in medicine. Science. 363: 810-812.
11. Topol, E (2019). Deep Medicine: How Artificial Intelligence Can Make Healthcare Human Again. New York, NY, Basic Books.
12. Gardner, RL, Cooper, E, et al. (2019). Physician stress and burnout: the impact of health information technology. Journal of the American Medical Informatics Association. 26: 106-114.
13. Rajkomar, A, Kannan, A, et al. (2019). Automatically charting symptoms from patient-physician conversations using machine learning. JAMA Internal Medicine. Epub ahead of print.

Friday, March 29, 2019

Data Science, Biomedical Informatics, and the OHSU Department of Medical Informatics & Clinical Epidemiology

(The following is reposted from Health, Data, Information and Action, the blog of the Oregon Health & Science University Department of Medical Informatics & Clinical Epidemiology.)

Data Science is a broad field that intersects many other fields within and outside of biomedicine and health, including biomedical informatics. Data science is certainly an important component of research and educational programs in the OHSU Department of Medical Informatics & Clinical Epidemiology (DMICE).

What exactly is data science? There are many methods, but one consensus is, “the multi-disciplinary field that uses scientific methods, processes, algorithms and systems to extract knowledge and insights from data in various forms, both structured and unstructured” [1].

The definition of data science is somewhat different from the definition of biomedical informatics, which is “the interdisciplinary field that studies and pursues the effective uses of biomedical data, information, and knowledge for scientific inquiry, problem solving and decision making, motivated by efforts to improve human health” [2].

Clearly there is overlap as well as complementarity. As noted by Payne et al., biomedical informatics deals with a broader spectrum of data and information tasks, focused not only on what is learned from data but also how that is applied in a broader sociotechnical context [2].

Many DMICE research programs focus on aspects of Data Science:
  • Re-use of data from EHR (William Hersh, Aaron Cohen, Steven Bedrick) – leveraging data in EHR to identify patients as candidates for research studies and signals for rare diseases (porphyria) [3]
  • Documenting genomic variation in leukemia (Shannon McWeeney) – allowing for repurposing of drugs [4]
  • Quality of data for clinical care and research (Nicole Weiskopf) – methods for insuring completeness and comprehensiveness of data for use in research, quality measurement, and other tasks [5]
  • Urinary microbiome in health and disease (Lisa Karstens) – identifying role of microbiome and how its genetics can be leveraged for diagnosis and treatment [6]
  • Use of ambient data to detect and manage clinician strain – Dana Womack, Paul Gorman [7]
DMICE educational programs include Data Science in many of their courses. Our Bioinformatics & Computational Biomedicine (BCB) major includes:
  • Data Harmonization and Standards for Translational Research - BMI 533/633 (Instructors: Melissa Haendel, Ph.D., Ted Laderas, Ph.D., Christina Zheng, Ph.D.)
  • Management and Processing of Large Scale Data –BMI 535/635 (Instructors: Michael Mooney, Ph.D., Christina Zheng, Ph.D.)
  • Computational Genetics -BMI 559/659 (Instructor: Shannon McWeeney, Ph.D.)
  • Bioinformatics Programming and Scripting - BMI 565/656 (Instructor: Michael Mooney, Ph.D.)
  • Network Science and Biology- BMI 567/667 (Instructor: Guanming Wu, Ph.D.)
  • Data Analytics –BMI 569/669 (Instructors: Brian Sikora, Delilah Moore, Ted Laderas, Ph.D.)
Our Health & Clinical Informatics (HCIN) major includes:
  • Introduction to Biomedical and Health Informatics - BMI 510/610 (Instructor: William Hersh, M.D.)
  • Analytics for Healthcare - BMI 524/624 (Instructors: Abhijit Pandit, M.B.A., Tracy Edinger, N.D.)
  • Clinical Research Informatics- BMI 523/623 (Instructor: Nicole Weiskopf, Ph.D., Robert Schuff)
We also have developed ample instructional materials in Data Science for other learners:

1. Donoho, D (2017). 50 years of Data Science. Journal of Computational and Graphical Statistics. 26: 745-766.
2. Payne, PRO, Bernstam, EV, et al. (2018). Biomedical informatics meets data science: current state and future directions for interaction. JAMIA Open. 1: 136-141.
3. Wu, S, Liu, S, et al. (2017). Intra-institutional EHR collections for patient-level information retrieval. Journal of the American Society for Information Science & Technology. 68: 2636-2648.
4. Tyner, JW, Tognon, CE, et al. (2018). Functional genomic landscape of acute myeloid leukaemia. Nature. 562: 526-531.
4. Weiskopf, NG, Bakken, S, et al. (2017). A data quality assessment guideline for electronic health record data reuse. eGEMS. 5(1): 14.
6. Karstens, L, Asquith, M, et al. (2016). Does the urinary microbiome play a role in urgency urinary incontinence and its severity? Frontiers in Cellular and Infection Microbiology. 6:78.
7. Womack, D. (2018). Subtle cues: Qualitative elicitation of signs of strain in the hospital workplace. PhD Dissertation, Oregon Health & Science University.

Friday, March 8, 2019

Have We Passed the High-Water Mark of Seamless Technology?

Have we passed the high point of technology being seamless and easy to use? I recently had to give a presentation to faculty that included showing a few slides, and I assumed I could just bring my MacBook Pro, with its HDMI port, and plug it into the projector that was in the conference room on my campus where the meeting was taking place.

It was not meant to be. Despite having the right cables, my computer would not sync to the projector. That led to the search for other options. Which then led us into dongle hell. Maybe we should try the VGA adapter also available in the room. But of course my newer-model Mac had USB-C and not the older standard Mac connector that is still prevalent, at least among the dongles.

We finally got it all to work, although not without a lot of wasted time. The episode also got me thinking and wondering if we have passed with high-water mark of technology working seamlessly. Are those golden days overs?

I used to feel like an old-timer when I would reminisce about how technology used to be so awful in the late 20th century. I am sure I got snickers from the younger crowd when I bemoaned the old days of having to connect to my email over telephone modems, and trying to figure out the right number of commas in the ATDT command set of telephone modems to time when the lines would ask for things like long-distance codes to be entered. I also remember the days of bringing overhead transparencies of presentations just in case my computer would not sync with the archaic projectors. I recall the days before USB drives when transferring files between computers was sometimes impossible, especially with the disappearance of floppy disks (and presentation file sizes exceeding their capacity).

I believe that the golden days peaked shortly after the start of the 21st century. I would marvel that computers always now seemed to connect to and sync to projectors. First with wired Ethernet and then with the emergence of wifi, we were no longer at the mercy of telephone modems and noisy phone lines (or needing to find Internet cafes). We could connect to our email and institutional servers without make long-distance phone calls. We could even talk on the phone from almost anywhere via Skype.

But now we seem to be regressing. Newer projectors have more resolution, but don’t always work with older computers. Dongle hell is worst for the Mac, but the PC world is not immune. We have multiple connector types for projects, and different types of USB. An additional wrinkle is the need for encryption, and we can move files that pose very small risk (e.g., Powerpoint presentations) easily from where we create them to where we need to present them.

Technology still is marvelous when it works seamlessly. Hopefully the proliferation of cables, connectors, and security protocols will not make the golden era a distant memory

Thursday, February 14, 2019

Summary of New ONC and CMS Notices of Proposed Rulemaking for Health IT

This week, on the eve of the Healthcare Information Management Systems Society (HIMSS) conference, the Office of the National Coordinator for Health IT (ONC) and Centers for Medicare and Medicaid Services (CMS) each dropped a Notice of Proposed Rule Making (NPRM) concerning functionality, interoperability, certification, and others aspects of electronic health record (EHR) use.

These rules are required as described in the 21st Century Cures Act, which was major legislation to modernize aspects of biomedical research and health IT that passed with wide bipartisan majorities. I saw the act as a chance to "clean up" some of adverse and unintended effects of the Health Information Technology for Clinical and Economic Health (HITECH) Act (aka, meaningful use).

The Web page for the ONC NPRM not only has the 724-page proposed rule, but also a set of nine readable and understandable fact sheets about the rule and another seven devoted to aspects of the information blocking rule. A slide presentation from the HIMSS conference has a number of nice figures that diagram aspects of big picture. The fact sheets and slide presentation present a nice segue to then word-search in the 724-page PDF to find specific details.

Here is my summary of the key points of the ONC rule:
  • A new US Core Data for Interoperability (USCDI) that adds provenance information, 8 types of clinical notes, additional demographic information, and pediatric vital signs to the former Core Clinical Data Set
  • APIs to access USCDI using FHIR, with a base set of 13 Resources and two specific data fields within the Patient Resource that must be supported
  • SMART Application Launch Framework, using the OAuth2 and OpenID standards
  • Conditions and maintenance of EHR certification, with additional pediatric-specific criteria
  • Requirement to be able to export data for a patient and for all patients (when provider changing EHRs)
  • Rules for information blocking and allowable exceptions
  • Allowable fees to support information exchange (but not to the patient to access their data)
  • Standards Version Advancement Process to allow developers to choose among the versions of standards currently approved by National Coordinator
  • No developer gag clauses that prohibit discussion and demonstration of problems
The CMS NPRM is a companion rule that goes into more detail on some aspects, per its headings, some of which include:
  • Patient Access Through Application Programming Interfaces (APIs)
  • Health Information Exchange and Care Coordination Across Payers
  • API Access to Published Provider Directory Data
  • Care Coordination Through Trusted Exchange Networks
  • Public Reporting and Prevention of Information Blocking
  • Provider Digital Contact Information
  • Advancing Interoperability in Innovative Models
Clearly these rules are a necessary recalibration of our EHRs and their data, aiming to make them more patient-centric, clinician-friendly, and responsive to the EHR marketplace. While I suspect some smart people will come up with some good ideas why one thing or another should be changed in the final rules, my interpretation of the NPRMs is that they pretty much hit the target. I am confident they will lead to improved systems and our ability to do good with the data.