Wednesday, October 6, 2021

Certification for the Rest of Informatics

After several years of planning, professional certification is coming to the rest of the informatics field, i.e., moving beyond just board certification for eligible physicians. While certification is somewhat easier to apply in the context of the physician board model, the American Medical Informatics Association (AMIA) has now rolled out the AMIA Health Informatics Certification (AHIC, formerly Advanced Health Informatics Certification). Those who are certified will be designated as ACHIP, the AMIA Certified Health Informatics Professional. A section of the AMIA Web site provides detailed on the certification, eligibility for it, applying for and taking the exam, and recertification.

While AHIC is open to all who have a master's or doctoral degree in health informatics or a related discipline, the certification process is not conferred upon initial completion of one's education. Rather, individuals also need to have completed qualifying work experience to be eligible for certification. This is different from some fields, such as medicine, including the clinical informatics subspecialty, where one takes the board certification exam shortly after completing formal training. There are a number of healthcare disciplines that require significant work experience for certification, such as some of the advanced certifications offered by the American Nurses Credentialing Center.

The qualifications for AHIC are listed in a table on the AHIC Web site. There are two tracks of eligibility. Track 1 is for those who have a graduate degree in a health informatics-related area, e.g., health informatics, biomedical informatics, nursing informatics, public health informatics, translational bioinformatics, etc. Track 2 is for those who have a graduate degree in a related field, e.g., health professions such as nursing, pharmacy, and medicine, and other fields such as computer science and public health. The work time required for those in Track 1 is 50-100% work time over the last four of six years or 20-49% time over the last six of eight years. The work time required for those in Track 2 is 50-100% work time over the last six of eight years or 20-49% time over the last eight of 10 years.

The certification process is being developed and managed by the Health Informatics Certification Commission (HICC), a 14-member commission that is part of AMIA yet has considerable autonomy from AMIA, especially with regards to AMIA's educational programs. The HICC is responsible for eligibility, examination development, and recertification requirements for AHIC.

The first offering of the certification exam is taking place this fall. The outline of exam topics follows  the health informatics workforce analysis commissioned by AMIA (Gadd, C.S., Steen, E.B., Caro, C.M., Greenberg, S., Williamson, J.J., Fridsma, D.B., 2020. Domains, tasks, and knowledge for health informatics practice: results of a practice analysis. J Am Med Inform Assoc 27, 845–852., just as the clinical informatics subspecialty exams now uses the complementary clinical informatics subspecialty workforce analysis for its exam blueprint.

The two questions someone enrolled in or contemplating seeking a degree in informatics will likely ask are: (1) Is this certification process for me? and (2) Will it benefit my career? Since this form of certification is new for professionals who work in informatics, the benefits at this time are unknown. The main drivers of the uptake will be employers who make hiring decisions that are influenced by job candidates having the certification. Similar to the clinical informatics subspecialty, we will probably see a gradual uptake of the AHIC over time. It may never be an absolute requirement for a job but it will be an important "feather in one's cap" when competing with others for a given position.

Tuesday, August 24, 2021

Scientific Rankings for the Informatics Professor

While I agree with those who argue that scientific rankings, especially based on bibliographic citation indicators, are limited in their measurement of a scientist's impact, I must admit a certain fascination with them. Perhaps that stems from my interest in dissemination and retrieval of scientific information generally. And perhaps also, because I enjoy writing and tend to measure well by these metrics.

I do show up in most rankings of my primary and related scientific fields. As I consider my primary scientific discipline to be biomedical informatics, I can report that I rank 57th on one list of researchers in the field. On another more focused list of those who work in medical informatics, I rank 14th, although my ranking falls to 33rd when the list is less focused (details below - also see [1,2]). I am also on a global list of the top 1000 computer science and electronics researchers, where I rank 932nd globally and 569th among Americans. On a more focused computer science list for the information retrieval field, I rank 23rd.

A description of how these rankings are calculated gives some perspective into my positions on them. All of these rankings make use of the well-known citation measure, the h-index, although one uses additional factors. The h-index is a measure of the number of one's publications that have been cited by at least that same number of publications. So for example, if one has 15 papers that have been cited 15 or more times, their h-index is 15. There are two main public sources of h-index values that are most commonly used, which give different results due to the way they are calculated. The two are Google Scholar and Scopus, the latter an arm of the scientific publishing conglomerate, Elsevier. The Google Scholar h-index is usually higher than the Scopus h-index due to the former including a wide variety of academic products, such as conference proceedings, books, non-peer-reviewed reports, and other publications on the Internet, whereas the latter is limited to journal publications. As the Google Scholar value is also generated automatically, it is more likely to contain erroneously included papers, especially when author names are ambiguous. My current Google Scholar and Scopus h-index values are, as of this writing, 75 and 46 respectively.

Obviously the h-index is related to the duration of one's career, and as citation patterns vary in different fields, one must compare the h-index of different individuals with caution. With those caveats, we can explore further my own ranks. The list of biomedical informatics researchers is maintained by Allison McCoy of Vanderbilt University. One concern about this list is that it contains a number of researchers who, although published in the biomedical informatics literature, do not primarily work in the biomedical informatics field. This list is generated from software developed by Jimmy Lin of University of Waterloo, who maintains the list of information retrieval researchers (and several other fields within computer science). The list of top worldwide computer science researchers is maintained by a Web site devoted to computer science research, Guide2Research.

An additional ranking in which I appear is one compiled by John Ioannidis of Stanford University and colleagues [1,2]. This analysis includes the top 100,000 scientists across all fields, with additional enrichment from those in the top 2% of their field but not in the top 100,000. Unlike the other sources, this analysis is fixed, with data through 2019 and taking more factors into account than just the h-index. A composite C-score is made up of six factors measured from citations through 2019 and excludes self-citations:

  • h19 (ns)    h-index as of end-2019
  • hm19 (ns)    hm-index as of end-2019
  • ncs (ns)    total citations to single authored papers
  • ncsf (ns)    total citations to single+first authored papers
  • npsfl (ns)    number of single+first+last authored papers
  • ncsfl (ns)    total citations to single+first+last authored papers

The rationale for this more complex measure is based on observations that (a) in some fields, many papers have vast numbers of authors, (b) these large numbers of authors give great weight to measures based purely on citations, (c) many Nobel laureates do not rate highly in simple citation measures such as h-index, (d) many of those who rank highly in simple citation measures have few or no first-authored or last-authored papers, and (e) Nobel laureates rank higher when more complex measures such as a C-score are employed.

In this cast of more than a hundred thousand, my C-score of 3.938 gives me an overall rank of 22,034, which is based on 241 papers published and 6109 citations to them through 2019. As noted above, I rank 15th among those whose primary field is medical informatics. There are also others for whom medical informatics is listed as their secondary field, and when combined with those for whom it is primary, my ranking is 34th(?). There is a separate ranking for those whose primary field is bioinformatics. 

I can also extract out all researchers in the ranking from my institution and its affiliates (Oregon Health & Science University, OHSU School of Medicine, Oregon National Primate Research Center, and Portland VA Medical Center) and note that I rank 49th out of 256 included in this list. (I am also pleased to note that 10 people from my department make it on to the overall OHSU list, including Roger Chou, Heidi Nelson, Mark Helfand, Joan Ash, Cynthia Morris, Paul Gorman, Rochelle Fu, Linda Humphrey, and Aaron Cohen.)

One interesting aspect of the Ioannidis et al. analysis is that I rank better using the composite score than just by my h-index. Based solely on the h-index, I would rank only 131st for OHSU and 37th in the primary medical informatics list. My C-score is improved by my relatively higher number of first-author and single-author papers, and citations to them. I also must have fewer co-authors on my papers than my colleagues at OHSU and in informatics, as I do better with the hm-index, which adjusts for the number of authors on a paper. At OHSU in particular, where I rank 49th overall and 131st by h-index, I rank 51st in hm-index, 37th in citations to single-authored papers, and 41st in citations to single- and first-authored papers. My 104 single- and first-authored papers rank me 22nd at OHSU. My data for the Ioannidis et al. analysis is available in a spreadsheet (Enjoy!).

On a final note, I am pleased to report that citation indices are a family affair for me. My daughter Alyssa Hersh, MD, MPH is currently a resident in Obstetrics & Gynecology at OHSU. She is also a rising researcher, and as of this writing has a Google Scholar h-index of 6 and a Scopus h-index of 4. I have no doubt she will surpass my current citation metrics long before she reaches my current age!


[1] Ioannidis, J.P.A., Klavans, R., Boyack, K.W., 2016. Multiple Citation Indicators and Their Composite across Scientific Disciplines. PLoS Biol 14, e1002501.

[2] Ioannidis, J.P.A., Boyack, K.W., Baas, J., 2020. Updated science-wide author databases of standardized citation indicators. PLoS Biol 18, e3000918.

Wednesday, July 14, 2021

Translational Artificial Intelligence: A Grand Challenge for AI

The potential for artificial intelligence (AI) to transform biomedicine and health is immense, yet at this time that potential remains largely unfulfilled. As I have noted in this blog over the years, there have been many impressive achievements in applying AI to biomedical and health data using clean and well-curated data sets, yet its use in every day medical care or health pursuits is modest. I have noted the reasons for this gap over the years in various postings, namely in the need to move capabilities beyond prediction to the ability to take action on them and the importance of showing actual clinical value, whether in leading to better health outcomes or care delivery. Failure to achieve these will relegate AI to the same fate as its first generation in the last century.

The premiere biomedical research agency in the US, the National Institutes of Health (NIH), has recognized the need to advance the science of AI in biomedicine. As such, the NIH has launched a new initiative, Bridge to Artificial Intelligence (Bridge2AI), which aims to "propel biomedical research forward by setting the stage for widespread adoption of artificial intelligence (AI) that tackles complex biomedical challenges beyond human intuition."

I applaud this program, and hope it will address the larger picture of translating current advances in AI into algorithms and systems that truly lead to improved health outcomes and care delivery. Along the way, it will hopefully address other issues related to AI, such as deploying AI ethically, focusing on solving important problems, and developing human expertise, not only of those who implement and evaluate AI systems but also the clinicians who employ them in their professional practice and patients who understand the benefits and limitations.

The Bridge2AI program recently hosted a virtual workshop series. One of these was devoted to enumerating grant challenges for AI in biomedicine. Participants were allowed to suggest grand challenges related to any aspects of the problems or solutions. I submitted a grand challenge focused on what I perceive is the need for translational AI, i.e., building on the successes in the "basic science" of showing the predictive value of algorithms to studying and ultimately deploying evidence-based AI in the real world. My grand challenge was one of 25 accepted by the meeting organizers for presentation to workshop attendees. We were allowed to use one slide, which is shown below.

To reiterate from my slide and presentation, I noted that most AI and machine learning research is still at the "basic science" stage of the biomedical research pipeline. Advances have been made with clean and well-curated data sets in simulated settings. Just as drugs and devices must progress from the lab to patient care, AI advances must do the same. In other words, moving from prediction to prescription and providing value. There are all sorts of potential ethical issues that must be addressed in the translational activities, such as balancing privacy protection vs. the public good, identifying and eliminating bias in data and algorithms, and ensuring the resulting actions do not exacerbate groups that have historically been discriminated against in health and healthcare. It is also critical that the "people perspectives" are addressed to develop the expertise in scientific as well as sociotechnical issues, including patients/consumers, clinicians, and researchers. In particular, appropriate education must be provided for clinical users and leaders to maximize value and minimize harm from these approaches. I am hardly the only person to hold these views, and a number of others have articulated the importance of AI providing real-world value in its application in biomedicine and health.[1-3]

One concern I have for Bridge2AI is that the first round of funding opportunities is focused on data generation projects. Generating high-quality, ethically-sourced, and relevant data is of course necessary, but is not sufficient. In fact, I have a hard time aligning what is proposed in my grand challenge with this initial funding opportunity. I hope that Bridge2AI will broad the focus of research and usher in translational research and ultimately demonstrates its true potential to improve human health.


1. Allen, B., Agarwal, S., Kalpathy-Cramer, J., Dreyer, K., 2019. Democratizing AI. Journal of the American College of Radiology 16, 961–963.
2. Wiens, J., Saria, S., Sendak, M., Ghassemi, M., Liu, V.X., Doshi-Velez, F., Jung, K., Heller, K., Kale, D., Saeed, M., Ossorio, P.N., Thadaney-Israni, S., Goldenberg, A., 2019. Do no harm: a roadmap for responsible machine learning for health care. Nat. Med. 25, 1337–1340.
3. Faes, L., Liu, X., Wagner, S.K., Fu, D.J., Balaskas, K., Sim, D.A., Bachmann, L.M., Keane, P.A., Denniston, A.K., 2020. A Clinician’s Guide to Artificial Intelligence: How to Critically Appraise Machine Learning Studies. Transl Vis Sci Technol 9, 7.

Friday, June 4, 2021

Virtual Graduation Message for 2021 OHSU Biomedical Informatics Graduates

I never imagined before 2020 that I would ever take part in a virtual graduation ceremony as I did in 2020, and now it is hard to believe that I am doing the same in 2021. As I noted last year, one of my favorite activities of the academic year is the Convocation and Hooding Ceremony, where we honor graduates of Oregon Health & Science University (OHSU), including those graduating from the OHSU Biomedical Informatics Graduate Program. Despite the pandemic over the last year and a quarter, we have 33 graduates this year from our PhD, Master of Science and Graduate Certificate programs. Since the programs inception in 1996, we have awarded 905 degrees and certificates to 815 people. I am very proud of all they accomplished during their time at OHSU and now beyond in various academic, industry, government, and other settings.

This year’s annual OHSU Convocation and Hooding Ceremony will take place virtually on Sunday, June 6, as noted in our department blog. Here is the transcript of my video message to the Class of 2021 graduates of the OHSU Biomedical Informatics Graduate Program:

It gives me great pleasure to welcome the 2021 graduates of the OHSU Biomedical Informatics Graduate Program to this year’s virtual Commencement ceremony. I never thought we would have a second virtual Commencement this year, but the pandemic is not yet fully behind us. With more and more people getting vaccinated each day, however, I am confident that next year’s ceremony will be in person, and I hope OHSU finds a way for those participating virtually the last couple years to take part in person in some way. If nothing else, we will celebrate next year at our annual DMICE banquet around graduation time that all alumni are always invited to attend. I do miss the pomp and circumstance of graduation, and getting to wear regalia and march in the procession.

As many of you know, the annual Commencement ceremony is an important event for me, as I enjoy every year celebrating the success of our graduates and their moving on to new paths in their lives. We have been awarding degrees and certificates from our program since 1998, and only once have I had to miss Commencement.

In any case, most of you are now moving from your studies into jobs where the contributions of informatics are more critical than ever. Just as the pandemic has exposed problems in our healthcare system, it has also exposed limitations in our data and information systems. It is critical for all informatics graduates, and everyone else in the informatics field, to keep improving how we use data and information, not only to overcome COVID-19 but also to improve health, health care, public health, and biomedical research generally. From bio- to imaging to clinical and public health informatics, the challenges have never been greater. I am confident that you all have the talent, and the knowledge and skills you have acquired in your studies, to meet those challenges.

I am pleased to report that with the 33 of you graduating today, our program has now awarded over 900 degrees and certificates dating back to 1998. These include 398 master’s degrees and 34 PhD degrees. Our graduates have achieved success in academia, industry, government, and just about every other place where informaticians work. Your success is one of the main aspects of our work that gives faculty and staff great satisfaction.

Let me close as always to remind you that even though you are moving on from OHSU and DMICE, we are still here for you and hope you will keep in touch with us as your careers develop and prosper.

Thursday, April 8, 2021

Response to NIH RFI: Comments and Suggestions to Advance and Strengthen Racial Equity, Diversity, and Inclusion in the Biomedical Research Workforce and Advance Health Disparities and Health Equity Research

In addition to its public health problems, the COVID-19 pandemic has exposed other fault lines in society, not the least of which is systematic racism that still pervades American society. A more equitable society would ensure diversity and inclusion in all aspects of life, including in healthcare and in biomedical research. The National Institutes of Health (NIH), the premier US federal agency that funds biomedical research, has recognized the need for its activities to be more inclusive of all Americans from every background. Not only must biomedical research reflect the health issues for the entire US population, its workforce should ideally reflect the ethnic and racial makeup of our larger society. The NIH recently issued a request for information (RFI), asking for Comments and Suggestions to Advance and Strengthen Racial Equity, Diversity, and Inclusion in the Biomedical Research Workforce and Advance Health Disparities and Health Equity Research. This posting contains the comments I submitted in response to this RFI.

While others will likely comment on the need for biomedical research itself to address health disparities and move toward health equity, it is equally important to address the needs of the biomedical research workforce that will contribute solutions to these problems. The NIH has already made a tremendous commitment to diversity and inclusion, including in building career pathways the biomedical research workforce, but additional efforts must be made to facilitate access to these programs by extramural researchers and leaders.

One example is the Building Infrastructure Leading to Diversity (BUILD) Initiative, which has a prominent program in our region.

Nonetheless, there are still challenges for engaging the potential future biomedical research workforce. My particular concern is how to increase diversity and inclusion among faculty of academic health science centers.

One of the challenges is explaining to young minds the opportunity and the work of biomedical research. While most young people are familiar with healthcare professionals - i.e., physicians, nurses, and pharmacists - fewer are familiar with the work and importance of researchers. Schools and communities themselves may not be aware of career opportunities. There should be resources committed, and easy-to-use tools made available, so that academic health science centers and others can disseminate information about careers and the rewarding work of biomedical research.

A second challenge is for researchers themselves to have the time to engage in such mentoring and teaching. As demands for productivity by biomedical research faculty in academic health science centers increase - i.e., keep grants funded and students taught - there is less time in their busy schedules for this critical activity. Such activity is also unlikely to "count" toward promotion or lead to that next grant. There should be standards for promotion committees in academic health science centers to require diversity and inclusion outreach. Of course, this must not be an "unfunded mandate," and instead be an activity that has committed time from institutions.

A third challenge is that success in biomedical research typically requires a graduate degree. As such, the road to college and then graduate or professional school is long and can be expensive. There must be pathways for students, especially for those of limited resources with few parental or other role models, to be helped through that long path. There should be opportunities provided, along with appropriate mentoring, for students to enable them to stay engaged during the long journey. Students should not only be given sustained exposure, but also be taught knowledge and skills along the way.

My own work is as an academic faculty in biomedical and health informatics, where I daily experience the satisfaction of research and teaching. While my field has made some strides in diversity and inclusion, it still has a long way to go to reflect our the racial and ethnic distribution of our larger society. Many who want to spend time engaging with future diverse researchers and professionals in the field need help in overcoming the above barriers. This leads to questions that must be answered:

  1. How do we engage with schools, community organizations, and others to expose high school and perhaps even younger students to biomedical research?
  2. How do we provide academic faculty with the protected time and academic credit for this work of critical importance?
  3. How do we develop pathways to sustain the interest and achievement for students, especially those from backgrounds that include little exposure to higher education?

We can and should require our academic health science centers and their faculty and others to engage with historically underrepresented groups in biomedical research, but make sure that they have the time and the tools, with milestones and outcomes measured, to achieve these goals. This should consist of:

  1. The availability of resources and tools to engage young minds in the possibilities for careers in biomedical research
  2. Expectations and protected time for existing faculty to devote effort to engaging with young students, including requirements to achieve promotion
  3. Developing pathways to sustain interest and achievement toward careers in biomedical research

By making diversity and inclusion efforts an expected activity of all biomedical research faculty and providing such faculty the resources and opportunities, we can achieve the shared aim of the biomedical research workforce and its activities reflecting the larger population of our country.

Thursday, February 25, 2021

A New OHSU Course in Applied Clinical Data Science and Machine Learning for Health & Clinical Informatics (HCIN) Students

I have written over the years about the need for all who work in biomedical and health informatics to have appropriate knowledge and skills in data science, machine learning (ML), artificial intelligence (AI), and related topics. I am now excited to announce that our OHSU Biomedical Informatics Graduate Program is launching a new course in Applied Clinical Data Science and Machine Learning for Health & Clinical Informatics (HCIN) majors.

The goal of this new course is not to provide students with the mastery of ML and AI tools and techniques; rather, it is to provide a conceptual understanding of their practical application in health and biomedicine. The course is not meant to be a substitute for the sequence of courses available in the other major in our program, Bioinformatics & Computational Biomedicine (BCB), whose offerings delve far more into the theory, mathematics, and programming of these topics and include:

  • BMI 551/651 - Statistical Methods
  • BMI 531/631 - Probability and Statistical Inference
  • BMI 543/643 - Machine Learning
  • BMI 525/625 - Principles and Practice of Data Visualization

The new HCIN course will be focused on applied data science and machine learning, with a focus on clinical data sets as well as clinical issues and challenges in their application. While the course will have some programming activity (requiring Python programming as a prerequisite), it will focus on a hands-on, high-level view of the different types of machine learning methods and their applications. It will also cover the topics of data management and selection, pitfalls in building and deploying models, and critical appraisal of clinical machine learning literature. The course will aim to provide an in-depth understanding for those who will work alongside experts who develop, build models, implement, and evaluate machine learning applications in health and clinical settings.

The textbook for the course will be: Hoyt, R. and Muenchen, R. (Eds.), 2019. Introduction to Biomedical Data Science, The course syllabus provides further details on the topics to be covered.

The content of the course will be based on a combination of what faculty and students believe is most important for a course like this. Among the topics that be included are:

  • Data sources - electronic health records, registries (e.g., N3C, AllOfUs), patient-generated, social media, public health
  • Data preparation (wrangling) - cleaning, quality analysis, feature selection, de-biasing
  • Exploratory data analysis - summaries, correlations, visualizations
  • Machine learning approaches and models - supervised, unsupervised, reinforcement, deep learning
  • Software and tools available
  • Common pitfalls and misunderstandings of applying machine learning
  • Critical appraisal of clinical machine learning literature
  • Ethical issues and challenges

The 3-credit course will be taught in the OHSU spring academic quarter, which runs from late March to early June. The lead instructors will be Steven Chamberlin, ND and myself, with other department faculty contributing. As with all courses in the HCIN major, it will be mostly online and asynchronous, with some option synchronous activities (which will be recorded for those not able to attend). This course will be different from to complementary to other data science-related courses in the HCIN major, including:

  • BSTA 525 - Introduction to Biostatistics
  • BMI 540/640 - Computer Science and Programming for Clinical Informatics
  • BMI 544/644 - Databases
  • BMI 524/624 - Data Analytics for Healthcare
  • BMI 516/616 - Standards/Interoperability in Healthcare
  • BMI 537/637 - Healthcare Quality
  • BMI 525/625 - Principles and Practice of Data Visualization

I will be excited to see how this course is accepted and how it evolves based on feedback of students and others. I suspect there will be interest beyond our graduate program.

Monday, February 22, 2021

Vaccinated and Vaccinating: The End May Be Near?

I was delighted to learn in early January that my institution, Oregon Health & Science University (OHSU), made the decision like many medical centers to offer the SARS-CoV-2 vaccine to all employees, not just those at the front line of care delivery. I received my first and second doses of the Pfizer vaccine on January 2nd and 23rd. I had some minor malaise the day after the second dose, but was thrilled to have received the vaccine.

I also decided that since I received an early dose, I would do everything I could to support the national and global effort to disseminate the vaccine. To that end, I have volunteered to work shifts at the OHSU Portland International Airport Vaccine Clinic. While I thought I might put my medical training to use giving injections, it turns out that the greater need was for registration and check-in personnel. I suppose it is most appropriate for the Chair of the informatics department to be checking in and scheduling follow-up appointments in Epic for those coming for their shots. But I actually enjoy the job I am doing at the site, interacting with people driving through the site and expressing gratitude they are able to get vaccinated. It is also nice to put on a friendly face for our university.

Overall, I feel a sense that the end may be near for the worst of this pandemic that has upended our lives. While the complete end will not come any time soon, and we will likely need to be vigilant about SARS-CoV-2 for years to come, I am hopeful that the vaccine rollout will continue at a strong pace and allow us to gradually resume more normal living. I am also encouraged that the COVID-19 numbers of cases, hospitalizations, and deaths are trending downward, and that we have new science-driven leadership in our federal government.

Looking ahead, I yearn to be around people at work, in social settings, and, yes, traveling. Regarding the latter, it has been almost a year since I have been on an airplane, although I am planning to visit my elderly stepfather, my last living adult relative, next month in Florida. He will have received his second dose a couple weeks before I visit.

There are many unanswered questions about what life will be like in the long run. Will work move to a more virtual arrangement? What will come of city centers that have been hurt by the pandemic and resulting economic and social upheaval? What will come of academic meetings and conferences, many of which probably could be done more virtually? Even though I spend a great deal of work time in front of a computer, I am still a social being. Social media has taken the sting off of the interpersonal isolation, but there is nothing like being around other people, and I am hopeful that much of that will eventually return. We will see as 2021 unfolds.