Keeping Evidence-Based in the Midst of a Pandemic

The Covid-19 pandemic requires urgent scientific knowledge about how to best diagnose, treat, and prevent the spread of the SARS-CoV-2 virus. This is at odds with the deliberate nature of evidence-based medicine (EBM), where it is important to use more deliberate methods to discern the best evidence.

Another challenge is to disseminate the results of research as quickly as possible. The availability of preprint servers and other modern Internet tools allow us to publish first and peer review later. But of course that raises worry that inadvertent error or even deliberate falsehoods might taint the quickly expanding evidence base.

How do we achieve a balance? We have already seen the downside of actions moving ahead of the science. Probably the best example of this is the drug hydroxychloroquine. While this drug may prove of value in preventing and treating SARS-CoV-2, it does have significant adverse effects, especially when taken in doses that exceed the normal therapeutic level. In addition, it is a drug whose availability for other diseases it treats, such as lupus, must be maintained for those patients.

Clearly hydroxychloroquine should be studied, but it should ideally be done in as controlled a way as possible, lest we not cause harm or generate false hope. We may not be able to perform classic double-blind, placebo-controlled randomized controlled trials, but we should still enroll and track patients in highly controlled manners. We cannot forget that this is a disease from which the majority of patients fully recover, so we need to make certain that improvements due to any treatment are not just due to normal recovery from the disease. There must be some sort of control group and a diligent follow-up to insure no missing data in control or experimental groups.

In my view, there are a number of critical questions to answer about SARS-CoV-2:

• How well do tests diagnose active infection with the disease?
• How well do tests diagnose serum antibodies indicating immunity?
• What treatments are available for the disease?
• Are there any preventive treatments for the disease, from drugs to immunizations?
• What is the best way to prevent spread in the general population?
• What is the best way to protect healthcare workers treatment patients with the disease?

All of these can be answered with the usual EBM methods of controlled studies that have served us well. They can also be augmented with large-scale data sources from which we are learning to do better observational studies. We can also carry out systematic reviews, with meta-analysis when appropriate, to collate the results of many studies.

Unfortunately, the deliberate pace of EBM must be balanced with the urgency to develop treatments, vaccinations, and methods to curtail spread of the virus. Likewise, the rapid publication of results on preprint servers and other sources must be followed with peer review and collation into systematic reviews and meta-analyses. Hopefully this will give us the best evidence based for treating and preventing this disease.

SARS-CoV-2: How Can I Help?

When the history of the global SARS-CoV-2 pandemic is written, the real heroes will be the frontline workers who cared for those in need and/or kept society functioning. This of course includes healthcare workers but also those working as first-responders and in public safety, or in grocery stores, gas stations, telecommunications infrastructure companies, and other essential businesses. They will certainly come off looking much better than political leaders or even “captains” of industry. I hope that whatever economic recovery plan is implemented that these workers will be appropriately rewarded and that society will have a better appreciation for the essential jobs they do.

It is natural for me to wonder how I can best contribute. As I “retired” from clinical practice some time ago, my skills as a clinician are probably not up to the task. However, there are probably some skills I can contribute, and I will consider those options going forward.

Fortunately there are some non-directly clinical contributions I can make, and these are keeping me busy here and now. One is teaching.  While society is first and foremost dealing with the crisis at hand, we cannot put all education on hold. The situation is particularly challenging for medical students. One might think that the current crisis gives them the opportunity to learn on the front lines. The reality, however, is that there is not enough personal protective equipment (PPE) to protect them. As such, we need to find other ways to maintain their learning trajectory.

A number of medical educators have come up with innovative approaches, and I have thrown my own contribution into the mix. As one who teaches a well-known virtual course that is an introduction to biomedical and health informatics, we are packaging up an offering that we intend to make available as a medical school elective. Because the course is mostly asynchronous, we can scale it up pretty quickly. I don’t just want to throw the materials out there, and still maintain some sort of interaction and connection with learners, but we can offer the course to many students (including those beyond medical students). We plan to launch the first offering to Oregon Health & Science University (OHSU) medical students next week.

I also have an opportunity to advance research related to SARS-CoV-2 in the form of organizing an information retrieval (IR) challenge evaluation. The goal of this retrieval challenge is both to help develop systems capable of identifying relevant information for the current pandemic, but also to scientifically study how retrieval methods can be quickly developed for such situations in the future. The task will follow the "Cranfield" evaluation procedures that are used in the Text Retrieval Conference (TREC) and related challenge evaluations.

This effort is made possibly by work of the Allen Institute for AI and some collaborators who have assembled an open dataset, the COVID-19 Open Research Dataset (CORD-19). This collection of biomedical literature articles currently contains over 40,000 articles and will be updated weekly. Some colleagues and I will be organizing an IR challenge for search engines that find relevant COVID-related articles within this collection. This challenge will provide:

• A benchmark set of important COVID-related queries (e.g., coronavirus risk factors, COVID-19 ibuprofen)
• A set of manual judgments for CORD-19 articles on these queries
• An ongoing leaderboard for comparison of IR systems 

We are even collecting candidate queries in a crowdsourcing manner by asking people to suggest them on Twitter using the hashtag, #COVIDSearch.

The current plan is to run the competition in weekly batches, where that week's snapshot of CORD-19 is used as the corpus and the results of systems participating in that batch are pooled for manual assessment. We will likely use the Kaggle platform to create a “leaderboard” of those whose methods are most effective. The challenge may in the future expand to more detailed tasks such as information-filtering, question-answering, fact-checking, and argument mining.

I make no pretensions that the work I am doing is in any way comparable to front-line healthcare and other essential workers, but I am glad that I can make these contributions that will keep education and research functioning during this tremendous worldwide crisis.

SARS-CoV-2: The Course Ahead

As the frequency of my postings in this blog has declined in recent years, I have noted several times that the blog started in the frenzied early days of the Health Information Technology for Economic & Clinical Health (HITECH) Act, which was part of the American Recovery and Reinvestment Act (ARRA) and that was instituted in an attempt to blunt the Great Recession of 2008. HITECH was part of ARRA, and of course gave us the big investment that has greatly expanded the adoption of electronic health records (EHRs). A small part of HITECH included investment to build the capacity of the health IT workforce.

Now, of course, we are headed into new economic recessionary times due to SARS-CoV-2, also known as Covid-19 as well as the Novel Coronavirus. Will this be ARRA 2.0?

Before I say anything about reactions to SARS-CoV-2, let me clearly state my sorrow for those most directly affected. Obviously the most sorrow is for those whose lives have been directly impacted by the disease it is causing and also by the disease's impact on their loved ones. There is also sorrow for what is happening to those whose lives are otherwise substantially affected, with threats to their livelihoods or other aspects of their ability to obtain food, shelter, and health care. There is also the impact for those on the front lines, of course in healthcare settings, but also in public safety, grocery stores, and other places of “essential” work. And to a lesser extent the rest of us, obviously minuscule compared to those directly impacted, but with major alterations to our daily lives.

With sorrow does come some opportunity for gratitude. While this is clearly impacting my life, at the end of the day, what I hold most dear - family, friends, and colleagues - are all still there and appreciated for their presence and support. We also owe gratitude for the global Internet, which enables us to keep connected by email, social media, and perhaps most importantly, videoconferencing. A decade ago, the bandwidth and reach of the Internet would not have allowed this level of connection. I am also grateful for my knowledge and experience in online teaching, and how I might put it to work keeping students and faculty connected during these trying times. The latter will likely be a major part of my work effort going forward.

I am certain I will much more to write about in the days ahead. While I had not hoped it would take a crisis to revitalize my blog, it will no doubt do so.