Physician and Medical Student Competence in AI Must Include Broader Competence in Clinical Informatics

A number of interesting papers on physician competencies in artificial intelligence (AI) have been published recently, and there is no question that all 21st century healthcare professions must have a thorough understanding of the benefits and limitations of AI that they are likely to use in their clinical work.(1-5)

One of my concerns, however, is that the focus on AI and not the larger issues of clinical informatics risks undermining not only a full understanding of the impact of AI, but also most of the other informatics-related knowledge and skills that are important to clinical practice. These include skills in using the electronic health record (EHR); accessing clinical knowledge using search systems; being facile with clinical decision support and health information exchange; protecting privacy and security, engaging patients, their data, and their devices; and applying data in tasks such as population health, public health, and clinical and translational research. At a minimum, these competencies provide a foundation of applying data, information, and knowledge to improve human health and healthcare delivery, but they also inform the application of AI in biomedicine and health.

About a decade ago, some colleagues and I published a paper outlining what we believed were the required competencies in clinical informatics in 21st century practice.(6) These competencies were then used to develop a curriculum in clinical informatics for our medical students.(7) While AI is now a prominent part of biomedicine and health, and a good deal more in society, the initial competencies have, in my opinion, stood the test of time.

There were originally 13 competencies in the list. In 2020, it became apparent that we needed to add an additional competency in machine learning, and in a textbook chapter (8) and blog post, we added that as a 14th competency. Now of course, it is probably better to use AI explicitly in that competency. As such, I present a new version of the list of competencies in clinical informatics for medical education, which of course applies to all health professions students and practitioners. (Textual version in the Appendix below.)

References

1. Ötleş, E., James, C.A., Lomis, K.D., Woolliscroft, J.O., 2022. Teaching artificial intelligence as a fundamental toolset of medicine. Cell Rep Med 3, 100824.
2. James, C.A., Wachter, R.M., Woolliscroft, J.O., 2022. Preparing Clinicians for a Clinical World Influenced by Artificial Intelligence. JAMA 327, 1333–1334.
3. Russell, R.G., Lovett Novak, L., Patel, M., Garvey, K.V., Craig, K.J.T., Jackson, G.P., Moore, D., Miller, B.M., 2023. Competencies for the Use of Artificial Intelligence-Based Tools by Health Care Professionals. Acad Med 98, 348–356.
4. Liaw, W., Kueper, J.K., Lin, S., Bazemore, A., Kakadiaris, I., 2022. Competencies for the Use of Artificial Intelligence in Primary Care. Ann Fam Med 20, 559–563.
5. Goodman, K.E., Rodman, A.M., Morgan, D.J., 2023. Preparing Physicians for the Clinical Algorithm Era. N Engl J Med.
6. Hersh, W.R., Gorman, P.N., Biagioli, F.E., Mohan, V., Gold, J.A., Mejicano, G.C., 2014. Beyond information retrieval and electronic health record use: competencies in clinical informatics for medical education. Adv Med Educ Pract 5, 205–212.
7. Hersh, W., Biagioli, F., Scholl, G., Gold, J., Mohan, V., Kassakian, S., Kerns, S., Gorman, P., 2017. From Competencies to Competence: Model, Approach, and Lessons Learned from Implementing a Clinical Informatics Curriculum for Medical Students, in: Health Professionals’ Education in the Age of Clinical Information Systems, Mobile Computing and Social Networks. Elsevier, pp. 269–287.
8. Hersh, W., Ehrenfeld, J., 2020. Clinical Informatics, in: Health Systems Science, 2nd Edition. pp. 156–170.

Appendix - Competencies in Clinical Informatics for Health Professions Education (textual form)

1. Find, search, and apply knowledge-based information to patient care and other clinical tasks
2. Effectively read from, and write to, the electronic health record (EHR) for patient care and other clinical activities
3. Use and guide implementation of clinical decision support (CDS)
4. Provide care using population health management approaches
5. Protect patient privacy and security
6. Use information technology to improve patient safety
7. Engage in quality measurement selection and improvement
8. Use health information exchange (HIE) to identify and access patient information across clinical settings
9. Engage patients to improve their health and care delivery though personal health records and patient portals
10. Maintain professionalism in use of information technology tools, including social media
11. Provide clinical care via telemedicine and refer patients as indicated
12. Apply personalized/precision medicine
13. Participate in practice-based clinical and translational research
14. Use and critique artificial intelligence (AI) applications in clinical care

More Evidence That We Need More Evidence for AI Interventions

In a previous post, I related the case of an excellent model that predicted hospital readmission yet when used in the context of real-world effort to reduce admissions was not able to lower the rate.

Some new studies highlight this scenario again of excellent models and systems that, when studied, do not show real-world benefit. A couple papers in Annals of Internal Medicine find a similar scenario for one of the earliest uses of artificial intelligence (AI) to demonstrate success, which is computer-aided detection (CADe) of polyps during colonoscopy results. A systematic review of previous clinical trials found that while there was an increased in detection of pre-cancerous adenomas but not of advanced adenomas and in higher rates of unnecessary removal of non-neoplastic polyps.[1]

The journal also featured a new randomized controlled trial (RCT) that showed no significant difference in advanced colorectal neoplasia detection rate (34.8% with intervention vs. 34.6% for controls) or mean number of advanced colorectal neoplasias detected per colonoscopy.[2]

An accompanying editorial notes the challenges in implementing AI in real world, which may impact RCT results, but we must build evidence base to support use.[3]

On a different clinical topic of predicting future trajectories in estimated glomerular filtration rate (eGFR) in adults with type 2 diabetes and chronic kidney disease, a new study in JAMA Network Open found that the new model excels over previous models in more accurate estimation of risk earlier in the disease course.[4] However, an accompanying editorial notes that while this model provides more accuracy, the benefit to those in this phase of the disease might be outweighed by "inappropriate avoidance of intravenous contrast, patient anxiety, and unnecessary testing with its associated costs."[5] What is really needed, the author notes, are clinical trials to validate use of the model.

The research into these clinical applications of AI is important, and we must carry out the "basic science" research of them. But then we must move on to the next step of clinical application and studies that evaluate such systems in clinical trials or other appropriate evaluation methods.

References

1. Hassan, C., Spadaccini, M., Mori, Y., Foroutan, F., Facciorusso, A., Gkolfakis, P., Tziatzios, G., Triantafyllou, K., Antonelli, G., Khalaf, K., Rizkala, T., Vandvik, P.O., Fugazza, A., Rondonotti, E., Glissen-Brown, J.R., Kamba, S., Maida, M., Correale, L., Bhandari, P., Jover, R., Sharma, P., Rex, D.K., Repici, A., 2023. Real-Time Computer-Aided Detection of Colorectal Neoplasia During Colonoscopy : A Systematic Review and Meta-analysis. Ann Intern Med. https://doi.org/10.7326/M22-3678

2. Mangas-Sanjuan, C., de-Castro, L., Cubiella, J., Díez-Redondo, P., Suárez, A., Pellisé, M., Fernández, N., Zarraquiños, S., Núñez-Rodríguez, H., Álvarez-García, V., Ortiz, O., Sala-Miquel, N., Zapater, P., Jover, R., CADILLAC study investigators*, 2023. Role of Artificial Intelligence in Colonoscopy Detection of Advanced Neoplasias : A Randomized Trial. Ann Intern Med. https://doi.org/10.7326/M22-2619

3. Shung, D.L., 2023. From Tool to Team Member: A Second Set of Eyes for Polyp Detection. Ann Intern Med. https://doi.org/10.7326/M23-2022

4. Gregorich, M., Kammer, M., Heinzel, A., Böger, C., Eckardt, K.-U., Heerspink, H.L., Jung, B., Mayer, G., Meiselbach, H., Schmid, M., Schultheiss, U.T., Heinze, G., Oberbauer, R., BEAt-DKD Consortium, 2023. Development and Validation of a Prediction Model for Future Estimated Glomerular Filtration Rate in People With Type 2 Diabetes and Chronic Kidney Disease. JAMA Netw Open 6, e231870. https://doi.org/10.1001/jamanetworkopen.2023.1870

5. Sanghavi, S.F., 2023. Modeling Future Estimated Glomerular Filtration Rate in Patients With Diabetes-Are There Risks to Early Risk Stratification? JAMA Netw Open 6, e238652. https://doi.org/10.1001/jamanetworkopen.2023.8652