Thursday, July 16, 2020

Updated Informatics.Health

For many years, I have had a portion of my Web site devoted to an introductory overview of the informatics field entitled, What is Biomedical & Health Informatics? I originally created this site to provide an answer to that question I was asked from time to time. I still maintain and keep it up to date both to still provide an overview of the field as well as demonstrate the technology we use in our virtual courses.

Last year I had an upgrade of sorts, snagging the new domain name, Informatics.Health. With the 2020 updating of my larger course that is offered in the American Medical Informatics Association (AMIA) 10x10 ("ten by ten") program, I have now updated the content of the What Is site.

The main part of the site is the nine lecture segments on the following topics:
  • What is Biomedical and Health Informatics? (1) (24:32)
  • What is Biomedical and Health Informatics? (2) (18:49)
  • A Short History of Biomedical and Health Informatics (22:30)
  • Resources for Field: Organizations, Information, Education (25:29)
  • Examples of the Electronic Health Record (EHR) (24:56)
  • Data Science and Artificial Intelligence (1) (14:15)
  • Data Science and Artificial Intelligence (2) (22:07)
  • Information Retrieval (Search) (23:18)
  • Information Retrieval Content (29:09)
One change this year is that the materials are only in HTML 5, dropping the use of Adobe Flash, which is being phased out at the end of this year. (The tool used to create the lectures is Articulate Studio 360.) The lectures can be viewed on just about any Web platform, and work fine on my iPhone and iPad.

The site also contains links to books, articles, organizations, and educational Web sites.

The materials on the site are freely available and have been used by many educators and others. An article from the American Medical Association (AMA) described their use by medical educators during the COVID-19 pandemic.

Sunday, July 5, 2020

The Informatics Professor Goes Solar

This summer we installed solar panels on our roof at home. The timing was good since we needed our roof replaced, which enabled us to install solar panels right on top of it. Many people tend to think of Portland, Oregon as a cloudy place, but the summers are mostly sunny and, above the 45th parallel, the days are long. Of course, even when it is cloudy, solar rays still shine down on the Earth (and our solar panels). 
A natural question is the economics of solar energy for a home and location like ours. They are surprisingly good. Last year, our electricity use averaged about 600 kilowatt-hours (kWh) per month, which averages to about 20 kWh per day. We have always used more electricity in the winter than the summer, perhaps due to the summers being mild and the days of winter being short. We opted to install a system that would aim to zero out our electric bill. We could have added additional capacity to account for an electric car, but we are not looking to buy a new car at this time.

The system includes a reversible meter, so when the panels exceed our electricity use, the excess goes into the Portland General Electric (PGE) grid. While the excess rolls over from month to month, it does not roll over years. So we will likely build up excess production over the summer that will be offset in the winter. We will see for sure when our PGE bills start rolling in.

Our 24 solar panels generate up to 7.68 kW of DC power and 5.76 kW when converted to AC power. The system includes an app that allows us to track the energy generated by the system. It has some nice reporting features that allow us to compare different days. The app does not track how much energy goes into the grid, although we can read that off our reversible meter. The app also allows us to have a public Web page so anyone can look at the data for our system. While the app has more data to show, the public Web page does allow viewing of daily electricity generation:

As the solar electricity is purportedly cheaper than that delivered by PGE, the system is estimated save about $34,000 over its lifetime. It doesn’t hurt that we will get a 26% federal tax credit this year and additional incentives from the state of Oregon. All in all, we believe it is a sound investment not only in our house, but also in the global energy future.

Our energy usage will also be reduced by the 6.5-inch R35 insulation under the new roof. This will be beneficial both with our electronic air conditioning in the summer and our gas heating in the winter.

Thursday, July 2, 2020

Kudos for the informatics Professor - Winter/Spring 2020 Update

Like many in the informatics field, the Informatics Professor has been very busy the last few months due to increased teaching and research activities taken on in response to the Covid-19 pandemic. As such, I have not had a chance to provide one of my occasional kudos postings of accomplishments until now.

Before Covid-19 struck, I was elected President of the International Academy of Health Sciences Informatics (IAHSI). The IAHSI is an international honorific society of leaders in informatics, and I look forward to assuming the Presidency in November, 2020.

Also before the pandemic, I was very busy with travel and talks:
Shortly after these talks, the Covid-19 pandemic emerged, and my travel came to an abrupt halt while my teaching and research activities accelerated. Due to the need for medical students displaced from clinical sites to have virtual learning activities, I gave several offerings of my introductory informatics course to both OHSU students (3 offerings to a total of 44 students) and non-OHSU students (8 offerings to a total of 178 students). Some educators and others also made use of the my What is Informatics? Web resource, which was featured both in an article as well as a list of resources for medical educators by the American Medical Association (AMA).

Another educational activity of note was OHSU’s hosting of the Informatics Training Conference for those holding biomedical informatics and data science training grants from the National Library of Medicine. The conference was held for the first time ever in a virtual format.

My research activities during this time mostly focused on the TREC-COVID information retrieval challenge, although I was also finishing up some papers (forthcoming soon!) and writing grant proposals for future research activities. We did publish some papers on TREC-COVID in both Journal of the American Medical Informatics Association as well as SIGIR Forum.

As always, I was busy writing both for scientific and other publications. In late 2019, I wrote an article about our informatics program for the publication of our local tech industry, Techlandia.

While the latter half of 2020 will be much prolific for publishing of book chapters and the new forthcoming fourth edition of my information retrieval textbook, I did publish in early 2020 an update of my chapter on clinical informatics in the second edition of the AMA textbook on Health Systems Science (Hersh W, Ehrenfeld J, Clinical Informatics, in Skochelak SE, Hammoud MM, Lomis KD, Borkan JM, Gonzalo JD, Lawson LE, Starr SR (eds.), Health Systems Science, 2nd Edition, 2020, 156-171).

While I hope to back off a bit over the summer, there is much in store for the rest of the year in both research and education.

Friday, June 5, 2020

Virtual Commencement Message for OHSU Biomedical Informatics Graduates

One of my favorite activities of the year is Commencement, where we honor graduates of Oregon Health & Science University (OHSU), including those graduating from the OHSU Biomedical Informatics Graduate Program. This year’s annual OHSU Commencement ceremony has been moved to a virtual format, and will take place on Sunday, June 7, 2020 at 10 am. This posting contains the transcript of my video message to the Class of 2020 graduates of the OHSU Biomedical Informatics Graduate Program:

It gives me great pleasure to welcome the 2020 graduates of the OHSU Biomedical Informatics Graduate Program to this year’s virtual Commencement ceremony. The annual Commencement ceremony is an important event for me, as I enjoy every year celebrating the success of our graduates and their moving on to new paths in their lives. We have been awarding degrees and certificates from our program since 1998, and only once have I had to miss Commencement.

This year was already going to be a different Commencement ceremony. I would have attended the main event for all graduates with you all, but then would have not attended the OHSU School of Medicine Graduate Programs portion. That is because I would instead be attending another Commencement event, namely the medical student commencement because this year, as many of you know, my daughter is graduating OHSU with MD and MPH degrees. I am very proud of her and excited that she will be starting her residency in Obstetrics & Gynecology at OHSU later this month.

Of course, now the entire Commencement is a different event for all of us, because of the Covid-19 pandemic and the need for the entire ceremony to be virtual. I was hoping with all the rearranging that I still might get to share this time with you all, but alas, all of the follow-on ceremonies, including graduate programs and medicine, are scheduled immediately after the main session.

So this year you will get this brief message from me. I will miss the pomp and circumstance of graduation, and getting to wear regalia and march in the procession. Hopefully things will be back to normal next year, and perhaps some of you can return to take part then.

In any case, many of you are now stepping from your informatics studies into jobs where the contributions of our field are more critical than ever. Just as the pandemic has exposed problems in our healthcare system, it has also exposed limitations in our information and data systems. It is the mandate for all informatics graduates, and everyone else in the informatics field, to keep improving how we use information and data, not only to overcome Covid-19 but also to improve biomedicine and health generally. From bio- to imaging to clinical and public health informatics, the challenges have never been greater. I am confident that you have the talent, and skills you have acquired in your studies, to meet those challenges.

I am pleased to report that our alumni now number 782 individuals with 872 degrees and certificates dating back to 1998. These include 374 master’s degrees and 31 PhD degrees. Our graduates have achieved success in academia, industry, government, and just about every other place where informaticians work. Your success in your work and life generally is one of the main aspects of our work that gives faculty and staff great satisfaction.

Let me close as always to remind you that even though you are moving on from OHSU, we are still here for you and hope you will keep in touch with us as your careers develop and prosper.

Tuesday, May 12, 2020

Staying Healthy in a Pandemic

I assume that most people in the world, like myself, are assessing what is their personal risk for complications from Covid-19, should they become infected by the virus. I am one of those people I would consider to be on the cusp of risk. Being over age 60 and with borderline hypertension controlled by diet and exercise, I am at the beginning of where the curve starts to ascend for those risk factors. However, I am also very health-conscious, which I like to believe gives me some mitigation against those risks.

Fortunately in this pandemic, I have been able to live quite healthfully. My wife and I have been eating very well, having at least one big salad per day and enjoying her excellent vegetarian cooking. Being confined mostly to home, I have also taken to slightly upping my running routine back to where it was 5-10 years ago, running about five miles every other day. (And of course I keep my social distancing while running, giving a wide berth around anyone else on the road or trail with me.) Between improving my healthy life-style and traveling less (where like most, I tend to eat more), I have actually lost about 5-6 pounds since the start of pandemic.

I fully acknowledge that none of my lifestyle actions will completely mitigate against the risks, were I to become infected. So naturally, I am still hopeful for better treatments and ultimately a vaccine. In the meantime, I hope for the kind of testing, tracing, and quarantining that other countries have implemented much better than my country. And although it seems like a distant dream, I aspire to someday return to my former slightly less healthy lifestyle that allowed me to enjoy so much of the rest of the world. 

Sunday, April 26, 2020

Loss of an Early Informatics Visionary

I am saddened to learn of the passing of Dr. Burton "Bud" Rose due to complications of Covid-19. Dr. Rose was the creator of the well-known UpToDate online medical information system, which is used by clinicians around the world. I played a tiny role in the development of UpToDate by programming its first search capability. In the late 1980s, I was a postdoctoral fellow in medical informatics at Brigham & Women's Hospital, working in the lab of my mentor, Dr. Robert Greenes. Dr. Rose, a kidney specialist, came to Dr. Greenes seeking help to add a searching capability to a collection of "cards" of information about kidney diseases he had collated in an Apple Hypercard Stack. My research had been focusing, then as now, on information retrieval (search) systems. It was relatively straightforward to connect a simple system I had programmed to index and retrieve from the information in the cards. It was a marvel at the time to be able to type in a few words and get medical information, years before the onset of the World Wide Web and Google.

I ultimately finished my fellowship and moved on to Oregon in 1990, and the development of UpToDate was taken over by a fellowship colleague, Dr. Joseph Rush, who stayed on the project for years as it matured into a commercial product that expanded to all of medicine. In 2008, UpToDate was acquired by the large publisher, Kluwer. I had not seen Dr. Rose in many years, but he continued to be a vibrant clinician and educator until his recent retirement.

UpToDate is still widely used and revered in medical settings around the world. I believe its real value is in its content. While its modern search functionality is excellent, what really draws clinicians to it is the quality of its content that can be used to make clinical decisions based on rapid access to high-quality information.

Wednesday, April 22, 2020

Virtual Informatics Course for Medical Students Progresses

My virtual informatics course for medical students is starting its third offering this week, and the uptake has been great. We hope to keep offering the course through the OHSU spring academic quarter, which runs through early June.

The primary impetus for the course is that medical students have been sidelined from clinical experiences due to the need to protect their health as well as conserve personal protective equipment (PPE) for physicians, nurses, and others taking care of patients.

The usual 10-week course has also been organized into a 4-week block format for medical students. Students are required only to complete the weekly multiple-choice assessments and not a term paper or final exam. The course has been offered not only to OHSU medical students but also to any medical student from any US allopathic or osteopathic medical school. External students register for the course through their own institutions, who send us lists of students to enroll in the course.

The medical student course has been offered in weekly waves. The first course started with 17 OHSU medical students. The second two offerings include 62 medical students from 11 different medical schools: Dartmouth College (2), Northwestern University (1), University of Iowa (6), University of North Dakota (2), University of Rochester (15), City University of New York School of Medicine (2), Emory University (12), University of Miami (3), Philadelphia College of Osteopathic Medicine (9), Quinnipac University (1), and Stony Brook University School of Medicine (9). We anticipate additional waves of students from additional medical schools over the next few weeks.

Thursday, April 16, 2020

TREC-COVID: A New Information Retrieval Challenge for Covid-19

Calling all information retrieval (IR) and biomedical informatics researchers interested in IR! My colleagues and I are pleased to announce  a new research challenge related to Covid-19. TREC-COVID aims to develop and evaluate methods to optimize search engines for the current and rapidly expanding number of scientific papers about Covid-19 and related topics. A group of information retrieval (IR) researchers from the Allen Institute for Artificial Intelligence (AI2), the National Institute of Standards and Technology (NIST), the National Library of Medicine (NLM), Oregon Health and Science University (OHSU), and the University of Texas Health Science Center at Houston (UTHealth) have organized the challenge. A press release and official Web site for the project have been posted. Although not official to the project, I am also maintaining a page about the project.

TREC-COVID applies well-known IR evaluation methods from the NIST Text Retrieval Conference (TREC), an annual challenge evaluation that evaluates retrieval methods with data from news sources, Web sites, social media, and biomedical publications. In an IR challenge evaluation, there is typically a collection of documents or other content, a set of topics based on real-world information needs, and relevance assessments to determine which documents are relevant to each topic. Different research teams submit runs of the topics over the collection from their own search systems, from which metrics derived from recall and precision are calculated using the relevance judgments.

The document collection for TREC-COVID comes from AI2, which has created the COVID-19 Open Research Dataset (CORD-19), a free resource of scholarly articles about COVID-19 and other coronaviruses. CORD-19 is updated weekly, although fixed versions will be used for each round of TREC-COVID. It includes not only articles published in journals but also those posted on preprint servers, including bioRxiv, medRxiv, and others.

Because the dataset (along with the world's corpus of scientific literature on Covid-19) is being updated frequently, there will be multiple rounds of the challenge, with later ones focused on identifying newly emerging research. There may also be other IR-related tasks, such as question-answering and fact-checking. The search topics for the first round are based on those submitted to a variety of sources and were developed by myself, Kirk Roberts of UTHealth and Dina Demner-Fushman of NLM. Relevance judgments will be done by people with medical expertise, such as medical students and NLM indexers. I am overseeing the initial relevance judging process, which is being carried out by OHSU medical students who are currently sidelined from clinical activities due to the Covid-19 crisis.

Saturday, April 11, 2020

The Easiest of Times, the Hardest of Times

To paraphrase Charles Dickens,  these Covid-19 times are the easiest and hardest of times.

For me personally, the Covid-19 crisis so far has been relatively easy. Because of this, I have gratitude and also note my fortunes could change at any time. So far, none of my immediate family, friends, or colleagues has become infected or fallen ill. We are comfortably ensconced in our house, have access to just about all of life's essentials, and can enjoy the outdoors, including my own running, with careful physical distancing. Spring is arriving, and the weather over the last few days has been wonderful.

Likewise, my work life has for the most part gone on as usual. I actually have extra work in managing for the future of my department in the emerging financial recession and its impact on my academic medical center. But since my life is already highly virtual, my work has been relatively easy to carry on. For many years, almost all of the teaching I have been doing is already online. My other academic work, especially my informatics research, is also mostly virtual. I have also used this opportunity to offer up additional virtual teaching to other programs within and outside my university, and have become involved in one of many Covid-related informatics research initiatives.

But these times are still extremely hard. It is sad to watch and read the news, and see the statistics. It is difficult to see those succumbing to the virus and that impact on their families and friends, especially since better early management could have prevented some of that. I cannot say enough words of gratitude I have for frontline workers who are the true heroes of this pandemic, from those in healthcare to those in public safety, grocery stores, and other essential businesses. It is also difficult to see the economic impact, especially on those who cannot easily convert to remote work like I can. I also have tremendous worry not only for the recovery of public health and economy as well as inadequacies it has exposed in our healthcare system and the larger social support that society must provide.

It is easy to express my usual optimism being in my situation. While we must honor and protect those who have been impacted by this pandemic, we must also think of how we must also restructure society to insure not only a better approach for crisis times but also when times are good yet not everyone can benefit. 

Thursday, April 2, 2020

A Virtual Course in Biomedical and Health Informatics for Medical Students

Medical students from around the US (and the world) have had their education displaced by the Covid-19 pandemic. In many places, there is either desire to keep them away from risk or to preserve personal protecting equipment (PPE) to physicians, nurses, and others directly involved in patient care. As such, the medical education community has worked to identify virtual educational experiences for medical students.

Our contribution is a virtual course in biomedical and health informatics. Readers familiar with my work will recognize the content of this course as emanating from the introductory course in the Oregon Health & Science University (OHSU) Biomedical Informatics Graduate Program. This course is also used in OHSU's offering as part of the American Medical Informatics Association 10x10 ("ten by ten") program. The syllabus for the course details on how medical schools can enroll their students.

We are implementing the course as a 4-week medical student elective, which is awarded 2 credits at OHSU. The course has about 40 hours of lecture, and we anticipate another 40 hours spent on discussion forums, multiple-choice self-assessments for each unit, and optional readings. The course is graded as pass-fail, and passing requires completion of all of 10 units and their self-assessments over the 4 weeks of the course. Due to high demand, we are limiting enrollment to students in US-based allopathic and osteopathic medical schools.

One new offering of the course will begin each week starting Monday, April 6. We will enroll as many students as we have in a single section, and make all of the content available to them for the duration of the 4 weeks. We will make use of the discussion forums built into our LMS to answer questions they have, and raise a few questions for them to answer. At the end of 4 weeks, the course will end, and those who have completed all of the work will receive a passing grade, which we will report back to the contact from each school.

We are asking each medical school handle student enrollment and credit themselves. In other words, we will make the course available through our learning management system (LMS) at OHSU, but we will ask each school to provide us a list of students to enroll and each will get a login to the course. After the course is done, we will report back to the schools on whether each student completed the course or not. We would like for medical schools that participate to handle giving students credit (probably through some sort of self-study elective).

We also prefer that there be a single point of contact for each school with which we communicate. To capture this information, we have created an online survey that asks for the point of contact (please use a university email address), estimated number of students (initially up to 20 per school - we may be able to accommodate more later), and preferred dates (which we may need to change to balance load). After the survey is completed, someone from our staff will contact the schools to work out the details.

In addition, for those interested in less than a full course on informatics, we have an open Web site that provides some of the materials and is being used by some medical schools.

Monday, March 30, 2020

Keeping Evidence-Based in the Midst of a Pandemic

The Covid-19 pandemic requires urgent scientific knowledge about how to best diagnose, treat, and prevent the spread of the SARS-CoV-2 virus. This is at odds with the deliberate nature of evidence-based medicine (EBM), where it is important to use more deliberate methods to discern the best evidence.

Another challenge is to disseminate the results of research as quickly as possible. The availability of preprint servers and other modern Internet tools allow us to publish first and peer review later. But of course that raises worry that inadvertent error or even deliberate falsehoods might taint the quickly expanding evidence base.

How do we achieve a balance? We have already seen the downside of actions moving ahead of the science. Probably the best example of this is the drug hydroxychloroquine. While this drug may prove of value in preventing and treating SARS-CoV-2, it does have significant adverse effects, especially when taken in doses that exceed the normal therapeutic level. In addition, it is a drug whose availability for other diseases it treats, such as lupus, must be maintained for those patients.

Clearly hydroxychloroquine should be studied, but it should ideally be done in as controlled a way as possible, lest we not cause harm or generate false hope. We may not be able to perform classic double-blind, placebo-controlled randomized controlled trials, but we should still enroll and track patients in highly controlled manners. We cannot forget that this is a disease from which the majority of patients fully recover, so we need to make certain that improvements due to any treatment are not just due to normal recovery from the disease. There must be some sort of control group and a diligent follow-up to insure no missing data in control or experimental groups.

In my view, there are a number of critical questions to answer about SARS-CoV-2:
  • How well do tests diagnose active infection with the disease?
  • How well do tests diagnose serum antibodies indicating immunity?
  • What treatments are available for the disease?
  • Are there any preventive treatments for the disease, from drugs to immunizations?
  • What is the best way to prevent spread in the general population?
  • What is the best way to protect healthcare workers treatment patients with the disease?
All of these can be answered with the usual EBM methods of controlled studies that have served us well. They can also be augmented with large-scale data sources from which we are learning to do better observational studies. We can also carry out systematic reviews, with meta-analysis when appropriate, to collate the results of many studies.

Unfortunately, the deliberate pace of EBM must be balanced with the urgency to develop treatments, vaccinations, and methods to curtail spread of the virus. Likewise, the rapid publication of results on preprint servers and other sources must be followed with peer review and collation into systematic reviews and meta-analyses. Hopefully this will give us the best evidence based for treating and preventing this disease.

Wednesday, March 25, 2020

SARS-CoV-2: How Can I Help?

When the history of the global SARS-CoV-2 pandemic is written, the real heroes will be the frontline workers who cared for those in need and/or kept society functioning. This of course includes healthcare workers but also those working as first-responders and in public safety, or in grocery stores, gas stations, telecommunications infrastructure companies, and other essential businesses. They will certainly come off looking much better than political leaders or even “captains” of industry. I hope that whatever economic recovery plan is implemented that these workers will be appropriately rewarded and that society will have a better appreciation for the essential jobs they do.

It is natural for me to wonder how I can best contribute. As I “retired” from clinical practice some time ago, my skills as a clinician are probably not up to the task. However, there are probably some skills I can contribute, and I will consider those options going forward.

Fortunately there are some non-directly clinical contributions I can make, and these are keeping me busy here and now. One is teaching.  While society is first and foremost dealing with the crisis at hand, we cannot put all education on hold. The situation is particularly challenging for medical students. One might think that the current crisis gives them the opportunity to learn on the front lines. The reality, however, is that there is not enough personal protective equipment (PPE) to protect them. As such, we need to find other ways to maintain their learning trajectory.

A number of medical educators have come up with innovative approaches, and I have thrown my own contribution into the mix. As one who teaches a well-known virtual course that is an introduction to biomedical and health informatics, we are packaging up an offering that we intend to make available as a medical school elective. Because the course is mostly asynchronous, we can scale it up pretty quickly. I don’t just want to throw the materials out there, and still maintain some sort of interaction and connection with learners, but we can offer the course to many students (including those beyond medical students). We plan to launch the first offering to Oregon Health & Science University (OHSU) medical students next week.

I also have an opportunity to advance research related to SARS-CoV-2 in the form of organizing an information retrieval (IR) challenge evaluation. The goal of this retrieval challenge is both to help develop systems capable of identifying relevant information for the current pandemic, but also to scientifically study how retrieval methods can be quickly developed for such situations in the future. The task will follow the "Cranfield" evaluation procedures that are used in the Text Retrieval Conference (TREC) and related challenge evaluations.

This effort is made possibly by work of the Allen Institute for AI and some collaborators who have assembled an open dataset, the COVID-19 Open Research Dataset (CORD-19). This collection of biomedical literature articles currently contains over 40,000 articles and will be updated weekly. Some colleagues and I will be organizing an IR challenge for search engines that find relevant COVID-related articles within this collection. This challenge will provide:
  • A benchmark set of important COVID-related queries (e.g., coronavirus risk factors, COVID-19 ibuprofen)
  • A set of manual judgments for CORD-19 articles on these queries
  • An ongoing leaderboard for comparison of IR systems 
We are even collecting candidate queries in a crowdsourcing manner by asking people to suggest them on Twitter using the hashtag, #COVIDSearch.

The current plan is to run the competition in weekly batches, where that week's snapshot of CORD-19 is used as the corpus and the results of systems participating in that batch are pooled for manual assessment. We will likely use the Kaggle platform to create a “leaderboard” of those whose methods are most effective. The challenge may in the future expand to more detailed tasks such as information-filtering, question-answering, fact-checking, and argument mining.

I make no pretensions that the work I am doing is in any way comparable to front-line healthcare and other essential workers, but I am glad that I can make these contributions that will keep education and research functioning during this tremendous worldwide crisis.

Saturday, March 21, 2020

SARS-CoV-2: The Course Ahead

As the frequency of my postings in this blog has declined in recent years, I have noted several times that the blog started in the frenzied early days of the Health Information Technology for Economic & Clinical Health (HITECH) Act, which was part of the American Recovery and Reinvestment Act (ARRA) and that was instituted in an attempt to blunt the Great Recession of 2008. HITECH was part of ARRA, and of course gave us the big investment that has greatly expanded the adoption of electronic health records (EHRs). A small part of HITECH included investment to build the capacity of the health IT workforce.

Now, of course, we are headed into new economic recessionary times due to SARS-CoV-2, also known as Covid-19 as well as the Novel Coronavirus. Will this be ARRA 2.0?

Before I say anything about reactions to SARS-CoV-2, let me clearly state my sorrow for those most directly affected. Obviously the most sorrow is for those whose lives have been directly impacted by the disease it is causing and also by the disease's impact on their loved ones. There is also sorrow for what is happening to those whose lives are otherwise substantially affected, with threats to their livelihoods or other aspects of their ability to obtain food, shelter, and health care. There is also the impact for those on the front lines, of course in healthcare settings, but also in public safety, grocery stores, and other places of “essential” work. And to a lesser extent the rest of us, obviously minuscule compared to those directly impacted, but with major alterations to our daily lives.

With sorrow does come some opportunity for gratitude. While this is clearly impacting my life, at the end of the day, what I hold most dear - family, friends, and colleagues - are all still there and appreciated for their presence and support. We also owe gratitude for the global Internet, which enables us to keep connected by email, social media, and perhaps most importantly, videoconferencing. A decade ago, the bandwidth and reach of the Internet would not have allowed this level of connection. I am also grateful for my knowledge and experience in online teaching, and how I might put it to work keeping students and faculty connected during these trying times. The latter will likely be a major part of my work effort going forward.

I am certain I will much more to write about in the days ahead. While I had not hoped it would take a crisis to revitalize my blog, it will no doubt do so.

Sunday, February 23, 2020

Adding a New Competency in Clinical Informatics for Medical Education

One of the most widely cited papers I have written in the last decade has been one on competencies in clinical informatics for medical education [1]. For the most part, these 13 competencies have stood the test of time, from knowing how to use the electronic health record and information retrieval systems as well as applying clinical decision support, patient privacy, personal health records, telemedicine, and more. All of these aspects of clinical informatics are essential skills for the 21st century clinicians.

But another area of required competence has come to the fore in recent years, which is the explosion of machine learning and artificial/augmented intelligence in medicine. While the impact of these in real-world clinical practice is still small, the long-term effect is likely to be substantial. Certainly clinicians should be familiar with the myriad of issues related to algorithms and models, including ethical concerns.

In the process of updating our chapter for the forthcoming 2nd edition of the textbook, Health Systems Science [2], my co-author Dr. Jesse Ehrenfeld and I took the opportunity to make this revision to the competencies by adding a 14th one:
14. Apply machine learning applications in clinical care
a. Discuss the applications of artificial/augmented intelligence in clinical settings
b. Describe the limitations and potential biases of data and algorithms

As with the original competencies, we encourage others to improve upon them. But it is also important to add this critical new one to the full set, which are listed below.


1. Hersh, WR, Gorman, PN, et al. (2014). Beyond information retrieval and EHR use: competencies in clinical informatics for medical education. Advances in Medical Education and Practice. 5: 205-212.
2. Skochelak, SE, Hawkins, RE, et al., Eds. (2017). Health Systems Science. New York, NY, Elsevier.

Appendix - Competencies in Clinical Informatics for Medical Education, circa 2020

1.    Find, search, and apply knowledge-based information to patient care and other clinical tasks
a.    Information retrieval/search - choose correct sources for specific task, search using advanced features, apply results
b.    Evaluate information resources (literature, databases, etc.) for their quality, funding sources, biases
c.    Identify tools to assess patient safety (e.g., medication interactions)
d.    Utilize knowledge-based tools to answer clinical questions at the point of care (e.g., text resources, calculators)
e.    Formulate an answerable clinical question
f.    Determine the costs/charges of medications and tests
g.    Identify deviations from normal (labs/x-rays/results) and develop a list of causes of the deviation

2.    Effectively read from, and write to, the electronic health record for patient care and other clinical activities
a.    Graph, display, and trend vital signs and lab values over time
b.    Adopt a uniform method of reviewing a patient record
c.    Create and maintain an accurate problem list
d.    Recognize medical safety issues related to poor chart maintenance
e.    Identify a normal range of results for a specific patient
f.    Access and compare radiographs over time
g.    Identify inaccuracies in the problem list/history/med list/allergies
h.    Create useable notes
i.    Write orders and prescriptions
j.    List common errors with data entry (drop down lists, copy and paste, etc.)

3.    Use and guide implementation of clinical decision support (CDS)
a.    Recognize different types of CDS
b.    Be able to use different types of CDS
c.    Work with clinical and informatics colleagues to guide clinical decision support use in clinical settings

4.    Provide care using population health management approaches
a.    Utilize patient record (data collection and data entry) to assist with disease management
b.    Create reports for populations in different healthcare delivery systems
c.    Use and apply data in accountable care, care coordination, and the primary care medical home settings

5.    Protect patient privacy and security
a.    Use security features of information systems
b.    Adhere to HIPAA privacy and security regulation
c.    Describe and manage ethical issues in privacy and security

6.    Use information technology to improve patient safety
a.    Perform a root-cause analysis to uncover patient safety problems
b.    Familiarity with safety issues
c.    Use resources to solve safety issues

7.    Engage in quality measurement selection and improvement
a.    Recognize the types and limitations of different types of quality measures
b.    Determine the pros and cons of a quality measure, how to measure it, and how to use it to change care

8.    Use health information exchange (HIE) to identify and access patient information across clinical settings
a.    Recognize issues of dispersed patient information across clinical locations
b.    Participate in the use of HIE to improve clinical care

9.    Engage patients to improve their health and care delivery though personal health records and patient portals
a.    Instruct patients in proper use of a personal health record (PHR)
b.    Write an e-message to a patient using a patient portal
c.    Demonstrate appropriate written communication with all members of the healthcare team
d.    Integrate technology into patient education (e.g., decision making tools, diagrams, patient education)
e.    Educate patients to discern quality of online medical resources (Web sites, apps, patient support groups, social media, etc.)
f.    Maintain patient engagement while using an EHR (eye contact, body language, etc.)

10.    Maintain professionalism through use of information technology tools
a.    Describe and manage ethics of media use (cloud storage issues, texting, cell phones, social media professionalism)

11.    Provide clinical care via telemedicine and refer patients as indicated
a.    Be able to function clinically in telemedicine/telehealth environments

12.    Apply personalized/precision medicine
a.    Recognize growing role of genomics and personalized medicine in care
b.    Identify resources enabling access to actionable information related to precision medicine

13.    Participate in practice-based clinical and translational research
a.    Use EHR alerts and other tools to identify patients and populations eligible for participation in clinical trials
b.    Participate in practice-based research to advance medical knowledge

14.    Apply machine learning applications in clinical care
a.    Discuss the applications of artificial/augmented intelligence in clinical settings
b.    Describe the limitations and potential biases of data and algorithms

Tuesday, December 31, 2019

Annual Reflections at the End of 2019

It has been customary for me to post an annual reflection at the of end of the year for this blog. The world of blogging, and my own blogging, have certainly changed over the years. This is probably due in part to social media reaching a level of maturity or, in the parlance of the Gartner Hype Cycle, the plateau of productivity. When I started this blog, blogging was relatively new. In parallel was the excitement of the HITECH Act, which certainly transformed the world of biomedical and health informatics.

But I still enjoy blogging, and appreciate the forum it provides me to speak to things of interest in informatics. This past year included one milestone, which was the blog passing the one-decade old milestone from the first posting on March 12, 2009. But my frequency of posting has gone down over the years. This is only my 15th post for 2019, down from a peak of 44 posts in 2013.

Still, 2019 was another excellent year, and I remain gratified to have a career I find rewarding. One number that has achieved relevance for me as the year ends is 2600. This number represents two things. One is the number of people who have completed the OHSU offering of the AMIA 10x10 course. The other is the number of people who follow me on Twitter. My gradual embrace of Twitter is noteworthy because I posted several years ago that I did not find much use for it. My social media use has generally settled into using Twitter for professional purposes and Facebook for mostly personal uses, although many of my Facebook friends are also professional colleagues, so there is overlap with professional activity there.

I also have much gratitude for all the other wonderful things in my life, including my family, my friends and colleagues, and my health. And as 2019 draws to a close, we now enter a new decade, the twenties. I could never imagined many of the things that happened, good and bad, in the 2010s, so it will be interesting to see how the twenties develop.

Tuesday, December 3, 2019

Eligibility for the Clinical Informatics Subspecialty - 2019 Update

I have been posting periodic updates of the eligibility for the physician clinical informatics subspecialty since the early days of the subspecialty, motivated by regular receipt of emails from physicians asking me questions about their individual eligibility. Rather than reply from the beginning each time, I replied with a link to the latest posting on eligibility in this blog and then instruct them to read the posting and write back to me with any questions specific to their situation. I also pointed out that my recommendations were just my interpretation of the rules, which were officially set by the American Board of Preventive Medicine (ABPM) for physicians of all specialties other than pathology, which were set by the American Board of Pathology (ABPath).

The very first posting was in 2013, which was then superseded by several minor updates. The overall eligibility rules have not changed much: As a subspecialty of all medical specialties, one must have a primary board certification in one of the 23 physician specialties that are recognized by the American Board of Medical Specialties (ABMS). Unfortunately, this excludes physicians who never achieved primary specialty certification or whose primary certification had lapsed. (Most board certifications now must be renewed every 10 years, although some of us trained in the era when boards such as the American Board of Internal Medicine [ABIM] granted lifetime certification.)

Perhaps the most significant update to my periodic postings came after 2016, when ABPM and ABPath extended the "grandfathering period" for an additional five years from 2017 to 2022. As with most new specialties and subspecialties, the clinical informatics subspecialty has a grandfathering period allowing one to become eligible to sit for the board exam and get certified through a certain level of practice or educational ("non-traditional" fellowship or master's degree) attainment.

With less than three years to go before the end of the grandfathering period, the window of opportunity for the practice or educational pathways is closing, especially for those who do not yet have any formal work or training in the field. It is unlikely that ABPM and ABPath will extend the grandfathering period beyond 2022. After that time, the only way to achieve board eligibility will be through a fellowship accredited by the Accreditation Council for Graduate Medical Education (ACGME).

For physicians who might be able to achieve eligibility via the Practice Pathway, my same advice from 2013 still holds: It will be more difficult, especially for mid-career physicians, to achieve board eligibility in or after 2023, when the only pathway to board certification will be to complete an ACGME-accredited fellowship. One other option for physicians will be the Advanced Health Informatics Certification being developed by AMIA. This will be a certification open to all who work in informatics but will be a pathway to certification for physicians who are not eligible for the ABMS clinical informatics subspecialty.

One fortunate happening since 2013 has been the marked improvement of the ABPM Web site, including its page describing eligibility for the clinical informatics subspecialty. (It was always somewhat ironic that the original Web site for the clinical informatics subspecialty was so poorly designed.) Another improvement of this page is the simplification of the explanation for becoming board-eligible, with all "grandfathering" now folded into the Practice Pathway. The site notes two options are available in the Practice Pathway (from which I quote):
  • Time in Practice: Three years of practice in Clinical Informatics is required. Practice time must be at least 25% of a Full-Time Equivalent (FTE) to be considered. Practice time need not be continuous, however, all practice time must have occurred in the five-year period preceding June 30 of the application year. Practice must consist of broad-based professional activity with significant Clinical Informatics responsibility. Fellowship activity that is less than 24 months in duration or non-ACGME accredited may be applied toward the practice activity requirement. The actual training must be described for any fellowship activity. Documentation of Clinical Informatics research and teaching activities may also be submitted for review.
  • Masters or PhD in Biomedical Informatics: Credit for completion of a 24 month Masters or PhD program in Biomedical Informatics, Health Sciences Informatics, Clinical Informatics, or a related subject from a university/college in the US and Canada, deemed acceptable by the Board (e.g. NLM university-based Biomedical Informatics Training) may be substituted for the Time in Practice option above.
So what are the options for clinical informatics board subspecialty eligibility in late 2019? Since the 2022 exam is now less than three years away, the Time in Practice pathway is closed unless one has already been in a qualifying position since mid-2019 or earlier. Fortunately, the Masters pathway is still open, and our online Masters program at Oregon Health & Science University (OHSU) fits the criteria of the second bullet above, namely our being part of a program that is a National Library of Medicine (NLM) university-based training program. Indeed, we have a number of physicians who have enrolled in the program with the intent of graduating by June 2022 and becoming board-eligible for the 2022 exam. Several of them have formed a cohort to work together toward that goal.

Saturday, November 16, 2019

Informatics Has Arrived ... in the World of Fiction

You know that your scientific field has arrived when it shows up in fiction. Informatics has now reached that point, as the field is featured in two new novels.

One is written by a recently retired informatician, Perry Miller, MD, PhD of Yale University. Dr. Miller has transitioned to becoming a novelist, authoring the book, Lethal Injection (Koehler Books, Virginia Beach, VA). In this murder mystery based in a hospital, IT systems play a significant role in the story, and one of the central characters has a master’s degree in informatics from OHSU. I won’t give away the rest of the story, but can say it was an enjoyable book to read.

In the other book, the characters from House of God, a famous novel from when I was in medical school in the 1980s, are reunited at Man's 4th Best Hospital, which is also the name of the book (Penguin Publishing Group). The characters are brought back together in an effort to defeat HEAL, the “Healthy Electronic Assistance Link” electronic health record foisted upon this flailing health system in an effort to improve its bottom line … and maybe also improve patient care. HEAL was developed by “electrical engineering grads, isolated out in Cheese Country, Wisconsin.” Anyone who knows anything about informatics knows the vendor for which those HEAL developers work.

Both books also deal with another problem in medicine, which is the consolidation of healthcare systems and resulting emphasis on the bottom line, sometimes to the detriment of care and well-being of clinicians. But both books are nice reads, and I won't say more to spoil their stories, other than to note it is interesting to see informatics come of age in fiction.

Monday, November 4, 2019

Moving FHIR from Aspirational to Operational

After years of giving lip service to standards, the health information/informatics community is now taking interoperability very seriously. The reason is obvious: we have spent a decade ramping up adoption of electronic health records (EHRs) in the US and elsewhere, and we have learned in hindsight that in the hurry to get systems implemented as quickly and easily as possible, inadequate attention was paid to data standards and interoperability. As a result, we have EHRs across the health system that do not easily talk to each other. The has compromised the hopes we have had for simple health information exchange (HIE) [1], not to mention the myriad re-uses of EHR data for research and quality assurance data [2].

The adoption of standards was also hindered by problems with the existing HL7 messaging standards, from the venerable but limited Version 2 to the hopelessly complex Version 3. Then, being the right solution at the right time, along came the Fast Healthcare Interoperability Resources (FHIR) standard [3,4].

Demonstrating a strong need for improved interoperability, the uptake across healthcare has been swift. FHIR received an early boost by being synergistic with the SMART app framework [5]. Another early enthusiast was ONC, which baked FHIR into the new rules mandated from the 21st Century Cures Act. (I view the health IT aspects of this legislation as a way to clean up the insufficient attention to interoperability in the original HITECH Act).

Other communities have jumped on the bandwagon as well. The new push to make quality measures easier to extract from the EHR, as opposed to requiring manual extraction from chart reviews, in the electronic Clinical Quality Measures (eCQMs) project.  The research community has joined as well, with the National Institutes of Health (NIH) calling for its use. Other research groups have started to develop tools, such as in natural language processing (NLP) pipelines [6] and the Clinical Data to Health (CD2H) data coordinating center for the Clinical & Translational Science Award (CTSA) program.

Other uptake of FHIR includes its use in clinical decision support tools. The value-based care community has jumped on board as well in the Da Vinci Project that focuses on managing and sharing clinical and administrative data.  There are even applications in the education arena, with the description of a tool recently developed to use SMART on FHIR in a case-based learning situation [7].

Despite all the excitement and achievement, the operational use of FHIR remains modest. This is not to argue that it will not achieve success across the healthcare system, but at the present time its use in real-world situations is small. But the enthusiasm shows that the needs it is intended to address are real, and that it has the potential to provide effective solutions for interopreability problems.

One example of a great start but need for more development involves the Apple Health app on my iPhone. I love to pull out my phone and show anyone who wants to see that I can download a good portion of my medical record from my institution's Epic EHR system to the Apple Health app, even the function that lets me show the FHIR resources in raw XML. But the reality is that at the present time, about all I can do with this app is show the presence of my data to people.

There are still some unanswered questions about making FHIR operational, such as:
  1. How will the FHIR approach scale up to large and diverse data types from the EHR and beyond?
  2. Will we be able to transform the unstructured data in notes and other parts of the record into the detailed structured form of FHIR?
  3. How will people manage the data that leaves the healthcare system, especially consumers who may not be savvy about their medical data they now possess on their phones and other devices?
We therefore need to think at the present time of FHIR as more aspirational than operational. That said, we need to leverage the widespread enthusiasm across the healthcare system to build on the current foundation to carry out the real work that must be done to reach the goals that everyone has for standards-based, interoperable data.


1. Dixon B. Health Information Exchange - Navigating and Managing a Network of Health Information Systems. Amsterdam, Netherlands: Elsevier; 2016.
2. Meystre S, Lovis C, Bürkle T, Tognola G, Budrionis A, Lehmann C. Clinical data reuse or secondary use: current status and potential future progress. In: Holmes J, Soualmia L, Séroussi B, editors. Yearbook of Medical Informatics. 262017. p. 38-52.
3. Benson T, Grieve G. Principles of Health Interoperability - SNOMED CT, HL7 and FHIR, Third Edition. London, England: Springer; 2016.
4. Braunstein M. Health Informatics on FHIR: How HL7's New API is Transforming Healthcare. New York, NY: Springer; 2018.
5. Mandel J, Kreda D, Mandl K, Kohane I, Ramoni R. SMART on FHIR: a standards-based, interoperable apps platform for electronic health records. Journal of the American Medical Informatics Association. 2016;23:899-908.
6. Hong N, Wen A, Shen F, Sohn S, Wang C, Liu H, et al. Developing a scalable FHIR-based clinical data normalization pipeline for standardizing and integrating unstructured and structured electronic health record data JAMIA Open. 2019: Epub ahead of print.
7. Braunstein M, Oancea I, Barry B, Darlington S, Steel J, Hansen D, et al. The development and electronic delivery of case-based learning using a Fast Healthcare Interoperability resource system. JAMIA Open. 2019: Epub ahead of print. 

Wednesday, September 18, 2019

Closing the Loops on Data Science and Informatics

One of the most highly viewed posts of this blog is a 2015 posting, What is the Difference (If Any) Between Informatics and Data Science. One critique I have had of data science is the focus of most work on only showing prediction and not implementing prescription. In other words, how do we take the predictive output in an ever-increasing number of areas of biomedicine and turn it into programs that actually improve outcomes, whether better patient care, improved healthcare delivery, or more effective research? Some recent publications bring this issue to light and show that we have some loops to close before we attain the value of data science in biomedicine.

A couple of recent perspective pieces bring this closure into light. One is from colleagues Philip Payne, Elmer Bernstam, and Justin Starren [1]. In a Perspective last year in JAMIA Open, they put forth a model that delineates the loop that must be closed, from the development of data science (and informatics) models and systems to the real-world informatics that most who work in the field are familiar with of implementing and evaluating systems with real users and organizations. A more recent paper from Lenert et al. notes that as predictive models are put into place and impact outcomes, they will necessarily impact those models, which will need to be adjusted to the new reality of their use [2].

One aspect of this first loop to be closed is how we study data science and machine learning interventions in actual clinical practice. A pair of recently published papers demonstrate how models and systems can be built and validated, and then assessed in the clinical real world. A first paper by Barton et al. develops and evaluates a model for predicting sepsis from patient vital designs [3]. Sepsis is a medical problem of continued significance while vital sign data is readily available. A subsequent paper by Shimabukuro et al. implements a randomized controlled trial in two medical intensive care units, finding a decrease in length of stay in the units from 13.0 to 10.3 days and a 12.4% reduction in in-hospital mortality [4].

Another recent study assessed the application of machine learning to detecting colonic polyps during colonoscopy [5]. While the machine learning system worked effectively, it was mostly effective at recognizing polyps that were unlikely to progress to cancer quickly, such as small adenomas and hyperplastic polyps. Nonetheless, recognizing such polyps improves the overall quality of colonoscopy exam.

A second loop that will need to be closed to achieve the vision of widespread generalized application of data science will be the generation of standardized EHR data for use across the healthcare system. A group of colleagues and I wrote about this in 2013 [6], as have many others, but some recent work documents aspects of this problem are still not solved. Two recent analyses show variations in how physicians [7] and healthcare organizations [8] document patient care, which may lead to variation in data that is not due to underlying differences in patients.

The need to close these loops show we are still in the early days of machine learning and predictive algorithms. While their impact in medicine will likely be enormous in the long run, there is still much work that will need to be done to optimize their data and how they are most effectively used.


1. Payne P, Bernstam E, Starren J. Biomedical informatics meets data science: current state and future directions for interaction. JAMIA Open. 2018;1:136-41.
2. Lenert M, Matheny M, Walsh C. Prognostic models will be victims of their own success, unless. . . Journal of the American Medical Informatics Association. 2019; Epub ahead of print.
3. Barton C, Chettipally U, Zhou Y, Jiangce Z, Lynn-Palevsky A, Le S, et al. Evaluation of a machine learning algorithm for up to 48-hour advance prediction of sepsis using six vital signs. Computers in Biology and Medicine. 2019;109:79-84.
4. Shimabukuro D, Barton C, Feldman M, Mataraso S, Das R. Effect of a machine learning-based severe sepsis prediction algorithm on patient survival and hospital length of stay: a randomised clinical trial. BMJ Open Respiratory Research. 2019;4(1):e000234.
5. Wang P, Berzin T, Brown J, Bharadwa S, Becq A, Xiao X, et al. Real-time automatic detection system increases colonoscopic polyp and adenoma detection rates: a prospective randomised controlled study. Gut. 2019; Epub ahead of print.
6. Hersh W, Weiner M, Embi P, Logan J, Payne P, Bernstam E, et al. Caveats for the use of operational electronic health record data in comparative effectiveness research. Medical Care. 2013;51(Suppl 3):S30-S7.
7. Cohen G, Friedman C, Ryan A, Richardson C, Adler-Milstein J. Variation in physicians' electronic health record documentation and potential patient harm from that variation. Journal of General Internal Medicine. 2019; Epub ahead of print.
8. Glynn E, Hoffman M. Heterogeneity introduced by EHR system implementation in a de-identified data resource from 100 non-affiliated organizations. JAMIA Open. 2019; Epub ahead of print.

Wednesday, August 21, 2019

An Information Retrieval Researcher’s Peer Review of Recent Studies of Search Engine Influence on Voting Behavior

A good part of my informatics research work over three decades has focused on the evaluation of search, also called information retrieval or IR. I have been amazed as the reach of search systems has become a mainstream part of our society, especially given that when I started, IR systems were only used by those who had computers with accounts from companies offering subscription search services.

Now, however, searching is ubiquitous. Indeed, it is almost impossible not to search, as it is offered in the address bar of most Web browsers. In addition, the name of one famous search engine, Google, has become a verb that synonymous with searching, i.e., Googling. Few of us can imagine a world without information on almost any topic being available nearly instantaneously.

It was therefore of interest this week when the President of the United States latched on to some research purporting that manipulation of Google was responsible for shifting three or more million votes to Hillary Clinton, which happens to be the amount of popular votes that she received over Donald Trump in the 2016 election (despite his narrow victory in the Electoral College).

This research has been put forth by Robert Epstein, PhD, who claims to be a liberal Democrat, as if that somehow indicates his analysis is not biased. Of course, one’s political views should not have any influence over the outcomes of their research.

Let’s look at Epstein’s multifaceted claims and the evidence supporting them from the standpoint of an IR researcher. First is the “finding” that Google manipulated search results to retrieve information “biased” toward Clinton. And second is that the retrieval of this information resulted in shifting of votes from Trump to Clinton.

The finding of manipulated search results comes from the paper posted as a PDF to Epstein’s Web site. As such, it is not peer-reviewed. The paper claims to show that in the run-up to the 2016 election and afterwards, 95 individuals, 21 of them of whom designated themselves as “undecided,” had their Google searches tracked and sent to a crowdsourcing site, Mechanical Turk, for rating as to whether they were biased toward Clinton or Trump. They eliminated searches from people who had Gmail accounts due to an unsubstantiated assertion that Google provided such users different results (which the company denies).

If I were sent this paper for peer review by an IR journal, I would ask the following: How did the researchers choose the individuals for the study? What evidence supports excluding those who had Gmail accounts? Who were the people on Mechanical Turk who did the ratings for the study? How were they instructed by the researchers to determine “bias?” I would certainly demand answers to questions like these before I would recommend acceptance for publication. The Methods section of the paper would need to be substantially expanded.

Let’s say, however, that the authors came back with acceptable answers to my questions, and the study were published. What about the second claim that this bias could lead to “manipulating” anywhere from 2.6-10.4 million votes in Clinton’s favor? The evidence for this comes from a paper that was published in a peer-reviewed journal, a prestigious one at that, the Proceedings of the National Academy of Sciences (PNAS). That study, published in 2015, looked at five randomized trials assessing the “search engine manipulation effect” (SEME).

These studies may be credible, but it is dubious whether they can be used to claim biased search results may have impacted voting in the US 2016 election. The first three experiments in the PNAS paper recruited individuals in the San Diego, CA area to rate who they might vote for the two candidates in the Australian Prime Minister election (chosen because most in San Diego would be unlikely to have prior knowledge). A fourth experiment replicated the first three with a national audience of individuals recruited from Mechanical Turk, while a fifth experiment recruited undecided voters to assess information about candidates in a local election in India. There should be no question that any kind of exposure to information can influence one’s decision about voting, although it would be questionable whether these sorts of results could be applied to a national US election where these same people would be bombarded by articles, reports, advertising, and other sorts of information, perhaps even Fake News promulgated by foreign entities on Facebook or Twitter.

Epstein fused the results of this research together to claim that biased search results moved several million votes in the direction of Clinton in the 2016 election. He took this conclusion to a receptive audience of Republicans in the US Senate. The outcome was predictable, with no skepticism whatsoever. And then came the crowning glory of it all, a Presidential tweet.

The mainstream fact checkers had a field day with these claims. Clearly one incompletely reported and probably highly flawed study, fused with another one showing that in some instances, search results can influence voting behavior, is hardly evidence that alleged bias by Google moved votes to Clinton in 2016. Here are some of their assessments:
The results of all this remind of the famous joke by comedian Stephen Colbert, who once noted, reality has a liberal bias. I do believe it is important and I strive to keep political biases out of our research. Even in my teaching, I aim to present opposing points of view, although not aiming to give equivalence to all points of view. But research like this needs to be called out for its thinly veiled political goals, and I suppose on that front, its “results" can be called successful.