tag:blogger.com,1999:blog-18017861869446304952024-03-16T11:50:48.342-07:00Informatics ProfessorThis blog maintains the thoughts on various topics related to biomedical and health informatics by Dr. William Hersh, Professor, Department of Medical Informatics & Clinical Epidemiology, Oregon Health & Science University.William Hersh, MDhttp://www.blogger.com/profile/08252786698451811645noreply@blogger.comBlogger389125tag:blogger.com,1999:blog-1801786186944630495.post-59436926074155710842024-02-28T08:13:00.000-08:002024-02-28T08:13:13.768-08:00Kudos for the Informatics Professor - 2023 Update<p>Periodically in this blog, lately once a year, I review all of my professional accomplishments, including honors, talks given, papers published, and more for the past year. Last year was a transitional year for me professionally, as I stepped down from two leadership positions as Chair of the Oregon Health & Science University (OHSU) <a href="https://www.ohsu.edu/school-of-medicine/medical-informatics-and-clinical-epidemiology">Department of Medical Informatics & Clinical Epidemiology (DMICE)</a> and from being Director of the <a href="https://www.ohsu.edu/school-of-medicine/medical-informatics-and-clinical-epidemiology/biomedical-informatics-graduate">OHSU Biomedical Informatics Graduate Program</a>. As seen in this post, however, my productivity did not miss a beat in 2023, and in fact I am enjoying my work as much as ever by <a href="https://informaticsprofessor.blogspot.com/2023/06/reports-of-my-retirement-are-greatly.html">being able to focus on teaching, research, and writing</a>.</p><p>I was awarded one honor in 2023 which was the <a href="https://now.ohsu.edu/member/post/dr-william-hersh-recognized-lifetime-vision-and-leadership/c30a901e-e84e-4023-ba3a-7c02d9931a1a">Lifetime Achievement Award from the HIMSS Oregon Chapter</a>.</p><p>I also gave a large number of invited talks, some in person and others virtual. Links to my slides and references, and for some, videos of the talks, are available on <a href="http://www.billhersh.info">my Web site</a>. The talks given include:</p><p></p><ul style="text-align: left;"><li><i>From the Longwood Medical Area to Oregon: An Informatics Career Journey</i> - Harvard Clinical Informatics Lecture Series (virtual), January 24, 2023</li><li><i>Introduction to Informatics: What You Should Know as a Health Services Researcher</i> - VA Health Services Research & Development Advanced Fellowship Program (virtual), March 7, 2023</li><li><i>Translational Artificial Intelligence (AI): The Need to Translate from Basic Science to Clinical Value</i> - University of Iowa Distinguished Biomedical Scholars Lecture Series, Iowa City, IA, March 9, 2023</li><li><i>Competencies and Curricula Across the Spectrum of Learners for Biomedical and Health Informatics</i> - University of Texas Southwestern Clinical Informatics Research Colloquium (virtual), May 18, 2023</li><li><i>Informatics Innovation: Quarter-Century of OHSU Leadership</i> - HIMSS Oregon Chapter Annual Conference 2023, Portland, OR, June 1, 2023</li><li><i>Biomedical and Health Informatics: An Essential Discipline for 21st Century Medicine</i> - Informatics Colloquium, Department of Pathology, Indiana University School of Medicine (virtual), July 19, 2023</li><li><i>Biomedical and Health Informatics: An Essential Discipline for 21st Century Medicine</i> - Department of Medicine Meet the Global Expert Webinar, University of Cape Town, South Africa (virtual), August 17, 2023</li><li><i>Artificial Intelligence: Implications for Health Professions Education</i> - Keynote Talk, Commission on Accreditation for Health Informatics and Information Management (CAHIIM) Summit on Higher Education (virtual), September 28, 2023</li><li><i>Artificial Intelligence in Medicine: Promise and Peril</i> - Grand Rounds, Department of Pathology and Laboratory Medicine, OHSU (virtual), October 4, 2023</li><li><i>ChatGPT and Other AI Tools for Medicine and Medical Education</i> - International Association of Medical Science Educators (IAMSE) Fall 2023 Webcast Seminar Series: Brains, Bots, and Beyond: Exploring AI's Impact on Medical Education (virtual), October 5, 2023</li><li><i>Artificial Intelligence: Implications for Health Professions Education</i> - Educator’s Collaborative, OHSU (virtual), October 18, 2023</li><li><i>Artificial Intelligence in Medicine: Promise and Peril</i> - 33rd Infectious Diseases Society of Oregon Annual Meeting, Salem, OR, October 26, 2023</li><li><i>Artificial Intelligence in Medicine: Promise and Peril</i> - Public Health Officers Caucus of Oregon (virtual), November 28, 2023</li><li><i>Artificial Intelligence: Implications for Informatics Education</i> - OHSU DMICE Conference, Portland, OR, December 14, 2023</li></ul><p></p><p>I also participated in a number of panels at meetings:</p><p></p><ul style="text-align: left;"><li><i>Current Clinical Evidence and Available Technology</i> - Voice AI Symposium, Bridge2AI-Voice Consortium, Washington, DC, April 19, 2023 (Panelist)</li><li><i>Program Sustainability</i> - Office of the National Coordinator for Health Information Technology (ONC) Public Health Informatics & Technology (PHIT) Workforce Program (virtual) June 20, 2023 (Panelist)</li><li><i>Strategies for Effective and Equitable Partnerships</i> - Third Meeting of the Data Science Initiative for Africa Consortium, Kigali, Rwanda, November 8, 2023 (Panelist)</li><li><i>Building Human and Data Capacity in the NIH Data Science for Africa Initiative</i> - AMIA Annual Symposium 2023, New Orleans, LA, November 13, 2023 (Panelist and Moderator)</li><li><i>Bridging Training Gaps through Voice: An Ethics-based Approach to Teaching Trustworthy AI</i> - AMIA Annual Symposium 2023, New Orleans, LA, November 14, 2023 (Panelist)</li><li><i>Generative Applications of Large Language Models for Medical Education and Knowledge Searching: Shall We Count on ChatGPT and Co.?</i> - AMIA Annual Symposium 2023, New Orleans, LA, November 15, 2023 (Panelist)</li></ul><p></p><p>I appeared on an <a href="https://podcasters.spotify.com/pod/show/hexailab/episodes/William-Hersh-on-Pitt-HexAI-e2d7ji4">episode of the Health and Explainable AI Podcast</a> from the University of Pittsburgh HexAI Research Laboratory.</p><p>I was awarded one new grant in 2023 but continued to be busy with my four existing grant projects. The new grant was an administrative supplement to an existing grant, the <a href="https://www.b2ai-voice.org/">Bridge2AI Voice as a Biomarker project</a>, to develop a <a href="https://www.b2ai-voice.org/voice-ai-summer-school.php">summer school for college undergraduates and students in clinical training</a>. My existing grants include a National Library of Medicine (NLM) <a href="https://reporter.nih.gov/search/YNHvkbM9G0Gow2yC3Cqwyw/project-details/10879792">R01</a>, NLM training grants for <a href="https://www.nlm.nih.gov/ep/GrantTrainInstitute.html">informatics predocs and postdocs</a> and for a <a href="https://www.nlm.nih.gov/ep/R25_program.html">college summer internship program</a>, another training grant in the <a href="https://dsi-africa.org/">Data Science Initiative for Africa</a>, and being part of the Skills and Workforce Development Module of the <a href="https://www.b2ai-voice.org/">Voice as a Biomarker project</a>.</p><p>I continued my teaching in 2023. As always, a major part of my teaching my introductory course in biomedical and health informatics, which is offered as <a href="https://ohsu.campusconcourse.com/view_syllabus?course_id=44944&public_mode=1">BMI 510/610 at OHSU</a>, the <a href="https://dmice.ohsu.edu/hersh/10x10.html">AMIA-OHSU 10x10 ("ten by ten") course</a>, and as <a href="https://dmice.ohsu.edu/hersh/minf705B709A.html">MINF 705A/709B, a medical student elective at OHSU</a>. I also gave my annual lecture to OHSU medical students in their first month of class, <i>Information is Different Now That You're a Doctor</i>, on August 25, 2023.</p>William Hersh, MDhttp://www.blogger.com/profile/08252786698451811645noreply@blogger.com0tag:blogger.com,1999:blog-1801786186944630495.post-34494931491847692602024-02-07T10:56:00.000-08:002024-02-07T10:59:27.930-08:00Translational AI: A Necessity and Opportunity for Biomedical Informatics and Data Science<p>How much of the hype for artificial intelligence (AI) that will truly impact health, healthcare, and research is an unknown. The potential benefits are unequivocal, from assistant patients pursue actions to improve their health to giving guidance to clinicians in diagnosis and treatment to helping researchers find information and devise new ideas to advance their research.</p><p>I have published an invited <a href="https://nlmdirector.nlm.nih.gov/2024/02/07/translational-ai-a-necessity-and-opportunity-for-biomedical-informatics-and-data-science/">post</a> in the <a href="https://nlmdirector.nlm.nih.gov/">National Library of Medicine (NLM) Musings from the Mezzanine Blog</a>, the blog of the Director of the NLM. I chose to update some of my past writings posted in this blog with a new discussion of what I call <i>translational AI</i>.</p><p>The tl;dr is:</p><p></p><ul style="text-align: left;"><li>The actual day-to-day use of clinical AI in healthcare is still modest, according to surveys.</li><li>While there are thousands of machine learning model papers that are published, and many systematic reviews of those model papers, there are a much small number, probably on the order of 100, randomized controlled trials (RCTs) of AI interventions in healthcare.</li><li>Of those RCTs, not all have resulted in positive outcomes and a number of them have risk of bias concerns.</li></ul><p></p><p>Clearly, as in all of healthcare, we cannot do RCTs on every permutation of model, implementation, setting, etc. of AI. However, we must treat AI the same way as any other tool we use in healthcare: <i>Show us the evidence</i>. Granted, evaluating the use of AI has plenty of differences from evaluating other interventions used in patient care, such as drugs and devices. It is difficult to conure a “placebo” for AI, and hard to perform controlled studies when AI, such as ChatGPT, is all around us.</p><p>Nonetheless, we can apply evidence-based medicine (EBM) to help inform its clinical use. The ideal way to do that is through randomized controlled trials (RCTs), or ideally systematic reviews of RCTs. As I note in the post, this is imperative not only for those of us who promote the use of AI and other biomedical and health informatics interventions, but also for students and trainees looking for projects to develop impactful research programs in their careers.</p>William Hersh, MDhttp://www.blogger.com/profile/08252786698451811645noreply@blogger.com0tag:blogger.com,1999:blog-1801786186944630495.post-8903689487562640232024-01-30T12:52:00.000-08:002024-01-30T12:53:57.808-08:00Whither Search? A New Perspective on the Impact of Generative AI on Information Retrieval (IR)<p>When I was putting the finishing touches on the 4th edition of <a href="http://irbook.info/">my textbook on information retrieval (IR, also known as search) in the domain on biomedicine and health in 2020</a>, I wondered whether the major problems in the field of IR were mostly solved. Retrieval systems such as Google for general Web searching and PubMed for the biomedical literature were robust and mature. One literally had the world’s written knowledge at their fingertips for general and biomedical topics from these systems respectively (even if paywalls did not always allow immediate access to the content).</p><p>There were certainly some areas of IR where additional work was needed and important, e.g., search over specific types of content such as social media or, in the case of my own research, <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7660955/">electronic health record (EHR) data and text</a>. There were also some nascent advances in the application of machine learning, although the gains in experimental results were more incremental than transformative.</p><p>But any staidness of IR was upended by the emergence of generally available generative artificial intelligence (AI) chatbots, based on large language models (LLMs), initially with ChatGPT and soon others to follow. Shortly thereafter came generative AI capabilities added to the two major Web search engines, Microsoft Bing and Google. All of a sudden, searching the Web was transformed in ways that most of us did not see coming.</p><p>I recently took advantage of the <a href="https://academic.oup.com/jamia/pages/call-for-papers-for-special-focus-issue">call for papers for a special issue devoted to ChatGPT and LLMs in biomedicine and health</a> of the flagship journal for the field of informatics, JAMIA, to write a perspective piece on why search is still important, even in the era of generative AI. At least for me, while the answer to my question is important in a search, it is also critical to know where the information came from. In addition, as I am commonly synthesizing my own knowledge and views on a topic, I do not just want a single generative AI answer to my question but rather the source articles and documents so I can compare and contrast different views and develop my own answer.</p><p>At the close of the paper, I do acknowledge that there may well be areas of IR where generative AI may have major impact going forward. I know that there is a lot of buzz around retrieval-augmented generation (RAG), although for many of the questions on which I search, I am much more interested in generation-augmented retrieval (GAR?). That is, how can generative AI methods improve the way we search to steer us to the kinds of authoritative, originally sourced information we seek to carry out our work?</p><p>The day before the article was published, a <a href="https://www.theregister.com/Author/Thomas-Claburn/">reporter</a> who came across my preprint <a href="https://www.theregister.com/2024/01/30/ai_is_changing_search/">wrote a piece on the impact of AI on search</a>, noting some of the issues I raise with regards to accuracy and authority for search in fields like medicine and in academia.</p><p>The paper itself has been published in JAMIA as an Advance Article, <a href="https://doi.org/10.1093/jamia/ocae014">Hersh W, Search still matters: information retrieval in the era of generative AI, Journal of the American Medical Informatics Association, 2024, ocae014</a>. Unfortunately, the open-access publishing fee for JAMIA is fairly steep ($4125), especially for a short perspective piece like this, but those wanting to <a href="https://arxiv.org/abs/2311.18550">read it can access the preprint that I posted</a>.</p>William Hersh, MDhttp://www.blogger.com/profile/08252786698451811645noreply@blogger.com0tag:blogger.com,1999:blog-1801786186944630495.post-66493333207798144632023-12-31T06:04:00.000-08:002023-12-31T06:08:19.210-08:00Annual Reflections at the End of 2023<p>Keeping with my tradition for this blog, I am ending this calendar year with my retrospective on the past year and some musings looking forward. The number of postings I make to this blog continues to decline each year. It is not that I do not have a lot to say, but rather I do not seem to have as much time to say it.</p><p>One thought has been to move to a different medium, such as a podcast. While I do listen to a fair number of podcasts, I still myself seem to prefer my words to be written. If nothing else, text seems more accessible. Another option would be to switch to something like a Substack newsletter, and I subscribe to several of those, but I have no interest in monetizing this blog, so Blogspot still seems like the easiest way to go.</p><p>In any case, the year 2023 was a transitional year for me. After serving as inaugural leader of both the Oregon Health & Science University (OHSU) <a href="http://www.ohsu.edu/informatics">Department of Medical Informatics & Clinical Epidemiology (DMICE)</a> and the OHSU <a href="http://www.ohsu.edu/informatics-education">Biomedical Informatics Graduate Program</a> for a couple decades, I <a href="https://informaticsprofessor.blogspot.com/2022/09/a-new-pathway-for-my-career-in.html">decided to step back from both leadership roles</a>. With both entities in good shape, but needing some reinvigoration, not to mention institutional reinvestment, the time seemed right to step back. There is good interim leadership, and my hope is that OHSU will choose to recruit strong permanent leadership in the year ahead.</p><p>Despite stepping back from leadership, I am still <a href="https://informaticsprofessor.blogspot.com/2023/06/reports-of-my-retirement-are-greatly.html">busy and gratified with my work</a>. I have had more time to devote to research and teaching. My research program in information retrieval (IR, also known as search) continues to be productive and well-funded, with several grants that will last through at least the next few years. Likewise, my teaching activity, mainly centered around my introductory course in biomedical and health informatics that is offered as a <a href="https://ohsu.campusconcourse.com/view_syllabus?course_id=44944&public_mode=1">graduate course</a>, <a href="https://dmice.ohsu.edu/hersh/10x10.html">continuing education course</a> (the famous "10x10"), and as a <a href="https://dmice.ohsu.edu/hersh/minf705B709A.html">medical student course</a>, also keeps me busy.</p><p>A former Dean-then-President of OHSU reminded me years ago that when you step above the department chair role in academic medicine, you mostly leave behind your academic work. It became clear to me this year that my passion and enthusiasm lie more in the intellectual work of informatics and less in building new programs. I did plenty in the latter over the years but now hope to focus the rest of my career on the intellectual work of the field.</p><p>And there is much new intellectual work in informatics and related areas to keep us busy. Another big transition for 2023 was the emergence of generative artificial intelligence (AI). Although generative AI had been around for a half-decade or so, a real turning point was the November 30, 2022 release of ChatGPT. There is clearly as much hype as there is reality, but there is no question its impact in the long run will be profound.</p><p>Generative AI impacts me in many ways. Clearly in education, it has caused teachers at all levels to reassess assignments and assessments of students. Almost everything in the assignments that I give students can be aided with ChatGPT. I will be writing more about this in the months ahead, but I agree with those who advocate we incorporate generative AI and not fight it or pretend it does not exist. It will likely require a combination of policy and students adhering to honor codes about how they do their work.</p><p>My research in the area of IR is also impacted by generative AI. With the 2020 publication of the <a href="https://link.springer.com/book/10.1007/978-3-030-47686-1">fourth edition of my textbook in the field</a>, I was beginning to wonder how much more research that IR needed. Clearly search is ubiquitous and a vital part of any intellectual work. While generative AI has large impacts for how we search for information, especially answers to questions, I <a href="https://arxiv.org/abs/2311.18550">note in a paper currently in preprint form</a> and hopefully accepted for publication soon, there is still a role for seeking sources of knowledge and knowing the authority of those who produce those sources.</p><p>One subject I wrote a great deal about in recent years but has faded some from the forefront is COVID-19. This virus is still with us and is <a href="https://informaticsprofessor.blogspot.com/2023/03/covid-19-three-years-on.html">likely to continue to be so for much time to come</a>. And the other impacts caused by the pandemic, namely the continuation of mostly remote work, may never revert to as it was before the pandemic. I am ambivalent about work that is mostly remote, even though I have been working that way for many years, especially with my travel. But I do appreciate the convenience of my morning commute being a walk down the stairs to my basement home office, although I also miss being around other people when working. Hopefully we will find a better balance in the future.</p><p>Heading into 2024, I do look forward to continuing this blog, and hopefully writing more. I also look forward to advances in the field of biomedical and health informatics and continuing to be a part of them.</p>William Hersh, MDhttp://www.blogger.com/profile/08252786698451811645noreply@blogger.com0tag:blogger.com,1999:blog-1801786186944630495.post-10016823404782140012023-09-15T04:45:00.006-07:002023-09-15T04:46:42.643-07:00Physician and Medical Student Competence in AI Must Include Broader Competence in Clinical Informatics<p>A number of interesting papers on physician competencies in artificial intelligence (AI) have been published recently, and there is no question that all 21st century healthcare professions must have a thorough understanding of the benefits and limitations of AI that they are likely to use in their clinical work.(1-5)</p><p>One of my concerns, however, is that the focus on AI and not the larger issues of clinical informatics risks undermining not only a full understanding of the impact of AI, but also most of the other informatics-related knowledge and skills that are important to clinical practice. These include skills in using the electronic health record (EHR); accessing clinical knowledge using search systems; being facile with clinical decision support and health information exchange; protecting privacy and security, engaging patients, their data, and their devices; and applying data in tasks such as population health, public health, and clinical and translational research. At a minimum, these competencies provide a foundation of applying data, information, and knowledge to improve human health and healthcare delivery, but they also inform the application of AI in biomedicine and health.</p><p>About a decade ago, some colleagues and I published a paper outlining what we believed were the required competencies in clinical informatics in 21st century practice.(6) These competencies were then used to develop a curriculum in clinical informatics for our medical students.(7) While AI is now a prominent part of biomedicine and health, and a good deal more in society, the initial competencies have, in my opinion, stood the test of time.</p><p>There were originally 13 competencies in the list. In 2020, it became apparent that we needed to add an additional competency in machine learning, and in a textbook chapter (8) and <a href="https://informaticsprofessor.blogspot.com/2020/02/adding-new-competency-in-clinical.html">blog post</a>, we added that as a 14th competency. Now of course, it is probably better to use AI explicitly in that competency. As such, I present a new version of the list of competencies in clinical informatics for medical education, which of course applies to all health professions students and practitioners. (Textual version in the Appendix below.)</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLlhlreURAlfHKLQ5aB4qislEPxZ3b8JM8AfrVfFRLPpfuqObL2zeKj-9ROdeKrpe1KcP_AjGl7PstqjF0C8svFT6JdCtEU3v5vF3KcIUP6EUveuYAQWoq_x2dFi14IKjI0bU3aoLw9xcdlUCK_IxGRMlhsI3trQmiL_RIUc00odIEdNE-qm3Sbdtf8anP/s1368/comptencies.png" style="margin-left: 1em; margin-right: 1em;"><img alt="Competencies" border="0" data-original-height="1368" data-original-width="840" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLlhlreURAlfHKLQ5aB4qislEPxZ3b8JM8AfrVfFRLPpfuqObL2zeKj-9ROdeKrpe1KcP_AjGl7PstqjF0C8svFT6JdCtEU3v5vF3KcIUP6EUveuYAQWoq_x2dFi14IKjI0bU3aoLw9xcdlUCK_IxGRMlhsI3trQmiL_RIUc00odIEdNE-qm3Sbdtf8anP/w245-h400/comptencies.png" width="245" /></a></div><p>References</p><div><ol style="text-align: left;"><li>Ötleş, E., James, C.A., Lomis, K.D., Woolliscroft, J.O., 2022. Teaching artificial intelligence as a fundamental toolset of medicine. Cell Rep Med 3, 100824.</li><li>James, C.A., Wachter, R.M., Woolliscroft, J.O., 2022. Preparing Clinicians for a Clinical World Influenced by Artificial Intelligence. JAMA 327, 1333–1334.</li><li>Russell, R.G., Lovett Novak, L., Patel, M., Garvey, K.V., Craig, K.J.T., Jackson, G.P., Moore, D., Miller, B.M., 2023. Competencies for the Use of Artificial Intelligence-Based Tools by Health Care Professionals. Acad Med 98, 348–356.</li><li>Liaw, W., Kueper, J.K., Lin, S., Bazemore, A., Kakadiaris, I., 2022. Competencies for the Use of Artificial Intelligence in Primary Care. Ann Fam Med 20, 559–563.</li><li>Goodman, K.E., Rodman, A.M., Morgan, D.J., 2023. Preparing Physicians for the Clinical Algorithm Era. N Engl J Med.</li><li>Hersh, W.R., Gorman, P.N., Biagioli, F.E., Mohan, V., Gold, J.A., Mejicano, G.C., 2014. Beyond information retrieval and electronic health record use: competencies in clinical informatics for medical education. Adv Med Educ Pract 5, 205–212.</li><li>Hersh, W., Biagioli, F., Scholl, G., Gold, J., Mohan, V., Kassakian, S., Kerns, S., Gorman, P., 2017. From Competencies to Competence: Model, Approach, and Lessons Learned from Implementing a Clinical Informatics Curriculum for Medical Students, in: Health Professionals’ Education in the Age of Clinical Information Systems, Mobile Computing and Social Networks. Elsevier, pp. 269–287.</li><li>Hersh, W., Ehrenfeld, J., 2020. Clinical Informatics, in: Health Systems Science, 2nd Edition. pp. 156–170.</li></ol></div><div><div>Appendix - Competencies in Clinical Informatics for Health Professions Education (textual form)</div></div><div><div><ol style="text-align: left;"><li>Find, search, and apply knowledge-based information to patient care and other clinical tasks</li><li>Effectively read from, and write to, the electronic health record (EHR) for patient care and other clinical activities</li><li>Use and guide implementation of clinical decision support (CDS)</li><li>Provide care using population health management approaches</li><li>Protect patient privacy and security</li><li>Use information technology to improve patient safety</li><li>Engage in quality measurement selection and improvement</li><li>Use health information exchange (HIE) to identify and access patient information across clinical settings</li><li>Engage patients to improve their health and care delivery though personal health records and patient portals</li><li>Maintain professionalism in use of information technology tools, including social media</li><li>Provide clinical care via telemedicine and refer patients as indicated</li><li>Apply personalized/precision medicine</li><li>Participate in practice-based clinical and translational research</li><li>Use and critique artificial intelligence (AI) applications in clinical care</li></ol></div></div>William Hersh, MDhttp://www.blogger.com/profile/08252786698451811645noreply@blogger.com0tag:blogger.com,1999:blog-1801786186944630495.post-83097014925643150302023-09-06T06:02:00.002-07:002023-09-06T06:05:19.746-07:00More Evidence That We Need More Evidence for AI Interventions<p>In a <a href="https://informaticsprofessor.blogspot.com/2023/07/lessons-about-clinical-ai-from.html">previous post</a>, I related the case of an excellent model that predicted hospital readmission yet when used in the context of real-world effort to reduce admissions was not able to lower the rate.</p><p>Some new studies highlight this scenario again of excellent models and systems that, when studied, do not show real-world benefit. A couple papers in <i>Annals of Internal Medicine</i> find a similar scenario for one of the earliest uses of artificial intelligence (AI) to demonstrate success, which is computer-aided detection (CADe) of polyps during colonoscopy results. A systematic review of previous clinical trials found that while there was an increased in detection of pre-cancerous adenomas but not of advanced adenomas and in higher rates of unnecessary removal of non-neoplastic polyps.[1]</p><p>The journal also featured a new randomized controlled trial (RCT) that showed no significant difference in advanced colorectal neoplasia detection rate (34.8% with intervention vs. 34.6% for controls) or mean number of advanced colorectal neoplasias detected per colonoscopy.[2]</p><p>An accompanying editorial notes the challenges in implementing AI in real world, which may impact RCT results, but we must build evidence base to support use.[3]</p><p>On a different clinical topic of predicting future trajectories in estimated glomerular filtration rate (eGFR) in adults with type 2 diabetes and chronic kidney disease, a new study in <i>JAMA Network Open</i> found that the new model excels over previous models in more accurate estimation of risk earlier in the disease course.[4] However, an accompanying editorial notes that while this model provides more accuracy, the benefit to those in this phase of the disease might be outweighed by "inappropriate avoidance of intravenous contrast, patient anxiety, and unnecessary testing with its associated costs."[5] What is really needed, the author notes, are clinical trials to validate use of the model.</p><p>The research into these clinical applications of AI is important, and we must carry out the "basic science" research of them. But then we must move on to the next step of clinical application and studies that evaluate such systems in clinical trials or other appropriate evaluation methods.</p><p>References</p><p>1. Hassan, C., Spadaccini, M., Mori, Y., Foroutan, F., Facciorusso, A., Gkolfakis, P., Tziatzios, G., Triantafyllou, K., Antonelli, G., Khalaf, K., Rizkala, T., Vandvik, P.O., Fugazza, A., Rondonotti, E., Glissen-Brown, J.R., Kamba, S., Maida, M., Correale, L., Bhandari, P., Jover, R., Sharma, P., Rex, D.K., Repici, A., 2023. Real-Time Computer-Aided Detection of Colorectal Neoplasia During Colonoscopy : A Systematic Review and Meta-analysis. <i>Ann Intern Med</i>. <a href="https://doi.org/10.7326/M22-3678">https://doi.org/10.7326/M22-3678</a></p><p>2. Mangas-Sanjuan, C., de-Castro, L., Cubiella, J., Díez-Redondo, P., Suárez, A., Pellisé, M., Fernández, N., Zarraquiños, S., Núñez-Rodríguez, H., Álvarez-García, V., Ortiz, O., Sala-Miquel, N., Zapater, P., Jover, R., CADILLAC study investigators*, 2023. Role of Artificial Intelligence in Colonoscopy Detection of Advanced Neoplasias : A Randomized Trial. <i>Ann Intern Med</i>. <a href="https://doi.org/10.7326/M22-2619">https://doi.org/10.7326/M22-2619</a></p><div>3. Shung, D.L., 2023. From Tool to Team Member: A Second Set of Eyes for Polyp Detection. <i>Ann Intern Med</i>. <a href="https://doi.org/10.7326/M23-2022">https://doi.org/10.7326/M23-2022</a></div><div><br /></div><div>4. Gregorich, M., Kammer, M., Heinzel, A., Böger, C., Eckardt, K.-U., Heerspink, H.L., Jung, B., Mayer, G., Meiselbach, H., Schmid, M., Schultheiss, U.T., Heinze, G., Oberbauer, R., BEAt-DKD Consortium, 2023. Development and Validation of a Prediction Model for Future Estimated Glomerular Filtration Rate in People With Type 2 Diabetes and Chronic Kidney Disease. <i>JAMA Netw Open</i> 6, e231870. <a href="https://doi.org/10.1001/jamanetworkopen.2023.1870">https://doi.org/10.1001/jamanetworkopen.2023.1870</a></div><div><br /></div><div>5. Sanghavi, S.F., 2023. Modeling Future Estimated Glomerular Filtration Rate in Patients With Diabetes-Are There Risks to Early Risk Stratification? <i>JAMA Netw Open</i> 6, e238652. <a href="https://doi.org/10.1001/jamanetworkopen.2023.8652">https://doi.org/10.1001/jamanetworkopen.2023.8652</a></div>William Hersh, MDhttp://www.blogger.com/profile/08252786698451811645noreply@blogger.com0tag:blogger.com,1999:blog-1801786186944630495.post-70303236858886969862023-07-17T05:09:00.002-07:002023-07-21T15:03:17.415-07:00Lessons About Clinical AI From a Randomized Controlled Trial on Hospital Readmissions<p>Our news ands science feeds are so filled these days with everything artificial intelligence (AI), from large language models to their impacts on society, that we may miss important studies on other informatics topics, some of which may have lessons for AI. This is the case for a recent randomized controlled trial (RCT) on a hospital readmissions initiative (Donzé, 2023) and an accompanying editorial putting it in larger perspective.(Wachter, 2023)</p><p>Some may remember about a decade ago, when “data analytics” was the rage, and health insurance payors were noting with alarm the growing rate of hospital readmissions. The cost and frequency of readmissions was highlighted in a study finding as many as 20% of hospital admissions were readmissions within a month of previous discharge.(Jencks, 2009) Before this, several hospital discharge planning programs had been studied and found to reduce readmissions.(Naylor, 1994; Coleman, 2006; Jack, 2009) This situation led the US <a href="https://www.cms.gov/">Centers for Medicare and Medicaid Services (CMS)</a> to implement the <a href="https://www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/AcuteInpatientPPS/Readmissions-Reduction-Program">Hospital Readmissions Reduction Program (HRRP)</a> as part of the Affordable Care Act. Starting in 2012, the HRRP required public reporting of readmission rates for three common diseases: myocardial infarction, heart failure, and pneumonia, with hospitals having unusually high rates of readmissions being penalized.</p><p>Around the time that the HRRP was implemented, the <a href="https://informaticsprofessor.blogspot.com/2011/02/hitech-improving-healthcare-through.html">Health Information Technology for Economic and Clinical Health (HITECH) Act</a> was incentivizing the adoption of the electronic health record (EHR). This provided new unprecedented sources of data, and every predictive analyst set out to find models that used EHR data to predict patients most likely to be readmitted, with the goal of identifying those who could presumably be followed more closely and have readmissions averted. Numerous studies were published using models based on EHR data to predict patients at risk for readmission.(Amarasingham, 2010; Donzé, 2013; Gildersleeve, 2013; Shadmi, 2015)</p><p>Despite the plethora of predictive models, few interventions have actually been undertaken that demonstrate improved outcomes of care. One study found that the use of a readmission risk tool intervention reduced risk of readmission for patients with congestive heart failure but not those with acute myocardial infarction or pneumonia (Amarasingham, 2013) Another observational study found that readmissions did decline with the implementation of the new rule initially and for the targeted diseases, but less so for other conditions.(Zuckerman 2016) Other have noted that the program has had marginal benefit (Ody, 2019) and redirecting resources that might be devoted to other quality improvement efforts.(Cram, 2022)</p><p>Earlier this year, an RCT was published that assessed a multimodal care intervention aimed at reducing readmissions (Donzé, 2023). Carried out in four medium-to-large teaching hospitals in Switzerland, the study implemented the best-known predictive model for risk of readmission yet found no benefit for an intervention that included it. As noted in the accompanying editorial, just because we can predict something does not mean we can necessarily do something about it.(Wachter, 2023)</p><p>Why is this RCT pertinent to AI? Mainly because just being able to predict diagnoses or outcomes is not enough. I have written about this myself in this blog over the years. Whether we are talking about <a href="https://informaticsprofessor.blogspot.com/2014/08/beyond-prediction-data-analyticsdata.html">predictive analytics</a>, <a href="https://informaticsprofessor.blogspot.com/2017/11/from-predictive-to-prescriptive.html">next-generation data science</a>, or <a href="https://informaticsprofessor.blogspot.com/2021/07/translational-artificial-intelligence.html">AI</a>, no matter how sophisticated our models or compelling our predictive abilities, we must demonstrate how these systems impact outcomes, whether improved patient health or healthcare system processes.</p><p>How do we demonstrate the value of AI in health and healthcare? First, we must implement these systems in the real world. There is a great deal being written about the promise and challenges for implementing AI in clinical settings.(Hightower, 2023) But even implementing AI in the real world is not enough. We must also demonstrate that AI can lead to better outcomes, whether improved health or treatment of disease of patients or better delivery of healthcare services. One perspective to think about this is the continuum of translational research. As with all biomedical advances, we start with the basic science, and demonstrating value in the laboratory, which in this case is the use of curated data sets. The next step is to implement systems in real-world healthcare or community settings. Clearly these are complex interventions.</p><p>Ultimately, however, we must demonstrate experimentally that health or healthcare is improved by the AI intervention. The best experimental evidence comes from controlled experiments, ideally RCTs. And granted, such trials may be more complicated than the classic RCT of comparing a medication versus a placebo. These RCTs may involve complex designs, and results may be difficult to interpret if the trial does not show benefit. But building the evidence base for AI is essential, and studies like this from Donzé et al. demonstrate that even the best predictive models may not translate into better outcomes.</p><p>References</p><p>Amarasingham, R., Moore, B.J., Tabak, Y.P., Drazner, M.H., Clark, C.A., Zhang, S., Reed, W.G., Swanson, T.S., Ma, Y., Halm, E.A., 2010. An automated model to identify heart failure patients at risk for 30-day readmission or death using electronic medical record data. Med Care 48, 981–988.</p><p>Amarasingham, R., Patel, P.C., Toto, K., Nelson, L.L., Swanson, T.S., Moore, B.J., Xie, B., Zhang, S., Alvarez, K.S., Ma, Y., Drazner, M.H., Kollipara, U., Halm, E.A., 2013. Allocating scarce resources in real-time to reduce heart failure readmissions: a prospective, controlled study. BMJ Qual Saf 22, 998–1005.</p><p>Coleman, E.A., Parry, C., Chalmers, S., Min, S.-J., 2006. The care transitions intervention: results of a randomized controlled trial. Arch Intern Med 166, 1822–1828.</p><p>Cram, P., Wachter, R.M., Landon, B.E., 2022. Readmission Reduction as a Hospital Quality Measure: Time to Move on to More Pressing Concerns? JAMA 328, 1589–1590.</p><p>Donzé, J., Aujesky, D., Williams, D., Schnipper, J.L., 2013. Potentially avoidable 30-day hospital readmissions in medical patients: derivation and validation of a prediction model. JAMA Intern Med 173, 632–638.</p><p>Donzé, J., John, G., Genné, D., Mancinetti, M., Gouveia, A., Méan, M., Bütikofer, L., Aujesky, D., Schnipper, J., 2023. Effects of a Multimodal Transitional Care Intervention in Patients at High Risk of Readmission: The TARGET-READ Randomized Clinical Trial. JAMA Intern Med 183, 658–668.</p><p>Gildersleeve, R., Cooper, P., 2013. Development of an automated, real time surveillance tool for predicting readmissions at a community hospital. Appl Clin Inform 4, 153–169.</p><p>Hightower, M., 2023. Is Medicine Ready for AI? N Engl J Med 388, e49.</p><p>Jack, B.W., Chetty, V.K., Anthony, D., Greenwald, J.L., Sanchez, G.M., Johnson, A.E., Forsythe, S.R., O’Donnell, J.K., Paasche-Orlow, M.K., Manasseh, C., Martin, S., Culpepper, L., 2009. A reengineered hospital discharge program to decrease rehospitalization: a randomized trial. Ann Intern Med 150, 178–187.</p><p>Jencks, S.F., Williams, M.V., Coleman, E.A., 2009. Rehospitalizations among patients in the Medicare fee-for-service program. N Engl J Med 360, 1418–1428.</p><p>Naylor, M., Brooten, D., Jones, R., Lavizzo-Mourey, R., Mezey, M., Pauly, M., 1994. Comprehensive discharge planning for the hospitalized elderly. A randomized clinical trial. Ann Intern Med 120, 999–1006.</p><p>Ody, C., Msall, L., Dafny, L.S., Grabowski, D.C., Cutler, D.M., 2019. Decreases In Readmissions Credited To Medicare’s Program To Reduce Hospital Readmissions Have Been Overstated. Health Aff (Millwood) 38, 36–43.</p><p>Shadmi, E., Flaks-Manov, N., Hoshen, M., Goldman, O., Bitterman, H., Balicer, R.D., 2015. Predicting 30-day readmissions with preadmission electronic health record data. Med Care 53, 283–289.</p><p>Wachter, R.M., 2023. The Disappointing Impact of Interventions to Prevent Hospital Readmissions. JAMA Intern Med 183, 668–669.</p><p>Zuckerman, R.B., Sheingold, S.H., Orav, E.J., Ruhter, J., Epstein, A.M., 2016. Readmissions, Observation, and the Hospital Readmissions Reduction Program. N Engl J Med 374, 1543–1551.</p>William Hersh, MDhttp://www.blogger.com/profile/08252786698451811645noreply@blogger.com0tag:blogger.com,1999:blog-1801786186944630495.post-87151769016216622792023-06-12T06:26:00.003-07:002023-06-12T06:26:36.951-07:00Please Share Your Slides When You Start Your Talk<p>I know that some people lament <a href="https://www.techtarget.com/whatis/definition/death-by-PowerPoint">death by Powerpoint</a>, but I find a speaker's slides generally valuable as both a listener and a speaker. Yes, many people (including me!) could do better with slides, but I find them useful. As a speaker, they help me keep track of everything I want to say, and drive home some points with visuals or reference lists. As a listener, I can keep track of what is being talked about, pursue any references I might be interested in further, and view graphics that often bring home points that words (especially spoken-only) do not.</p><p>My general practice when giving a talk is to post a PDF of my slides to <a href="http://www.billhersh.info">my Web site</a>, often with a list of references cited in the talk, and then tweet out a link to the PDF. I also send the PDF to the sponsor of my talk for distribution to those attending. That way, those listening to me do not need to be snapping pictures with their phones or writing things on the slides down when they want to remember something on one of my slides.</p><p>Based on my practices, I find it frustrating when speakers do not share their slides. Sometimes they provide them later, which is still far from ideal, but some speakers never provide them at all. When these happen, you see many listeners in the room taking pictures of slides they want to remember.</p><p>It would be much easier if all speakers would share at least a PDF of their slides right at the start of their talks. There really cannot be anything so proprietary, especially in an academic talk, that cannot be shared when those in the audience are snapping pictures and, as is sometimes the case, posting the photos to social media.</p><p>I hope that speakers I will be listening to in the future will adopt this advice. I certainly plan to keep my present practice in this regard.</p>William Hersh, MDhttp://www.blogger.com/profile/08252786698451811645noreply@blogger.com0tag:blogger.com,1999:blog-1801786186944630495.post-50152728546849933332023-06-06T05:50:00.004-07:002023-06-06T05:50:57.373-07:00Reports of My Retirement Are Greatly Exaggerated<p>As most of my world knows, I stepped down as Chair of the <a href="http://www.ohsu.edu/informatics">Department of Medical Informatics & Clinical Epidemiology (DMICE)</a> at <a href="https://www.ohsu.edu">Oregon Health & Science University (OHSU)</a> at the end of last year. In addition to <a href="https://now.ohsu.edu/member/post/department-medical-informatics-clinical-epidemiology-leadership-transition/940cf1c6-85f6-469a-ba75-27e9b101864b">announcements made by OHSU</a>, I let my world know with postings <a href="https://informaticsprofessor.blogspot.com/2022/09/a-new-pathway-for-my-career-in.html">in this blog</a>, on social media, and via some email lists.</p><p>I was gratified by the many nice accolades that I received but one thing was unsettling. This was those who wrote to congratulate me on my retirement. It was unsettling because I wanted to make explicit that I was not retiring, and instead was passing on the leadership to others, initially to interim leaders from the department and eventually culminating in a search for a new permanent Chair. In the meantime, I would continue the teaching, research, writing, and mentoring activities that I enjoy so much.</p><p>Nearly a half-year in to my new status, I am indeed enjoying it. I still wake up each work day and get to do the intellectual work of informatics that I love most. My circumstance brings to mind a famous quote, <a href="https://www.mentalfloss.com/article/562400/reports-mark-twains-quote-about-mark-twains-death-are-greatly-exaggerated">yet which he probably never said</a>, by Samuel Clemons (Mark Twain) that reports of his death were greatly exaggerated.</p><p>I am therefore please to state that reports of my retirement are indeed exaggerated. I hope to continue working and be productive in the field for many more years. In fact, my plate is as full as ever, and my main work going forward will continue on the teaching, research, and funded grant projects that currently keep me busy.</p><p>My "plate" includes, among other things, the following major activities:</p><p></p><ul style="text-align: left;"><li>Introductory course – I have always enjoyed teaching an introductory course in <a href="https://ohsu.campusconcourse.com/view_syllabus?course_id=38420&public_mode=1">our graduate program as course BMI 510</a>, in the <a href="https://dmice.ohsu.edu/hersh/10x10.html">AMIA 10x10 ("ten by ten") program</a>, and for <a href="https://dmice.ohsu.edu/hersh/minf705B709A.html">OHSU medical students as MINF 705B/709A</a>. I also enjoy teaching some more advanced topics as well.</li><li>NLM R01 – Semi-structured Information Retrieval in Clinical Text for Cohort Identification, 2021-2026 (Grant R01LM011934). A research grant to developing test collections and methods for applying information retrieval methods to electronic health record data.</li><li>NIH <a href="https://dsi-africa.org/">Data Science Initiative for Africa</a>, Computational Omics and Biomedical Informatics Program (COBIP), 2021-2026 (Grant U2RTW012131). A collaboration with the University of Cape Town to build human and data set capacity in Africa.</li><li>NLM <a href="https://www.nlm.nih.gov/ep/GrantTrainInstitute.html">T15 Training Grant</a> – Research Training in Biomedical Informatics and Data Science at Oregon Health & Science University, 2022-2027 (Grant T15LM007088). OHSU's <a href="https://www.ohsu.edu/school-of-medicine/medical-informatics-and-clinical-epidemiology/fellowships">informatics training grant to fund PhD students and postdocs</a>.</li><li>NLM R25 <a href="https://www.nlm.nih.gov/ep/R25_program.html">Short-Term Research Education Experiences to Attract Talented Students to Biomedical Informatics/Data Science Careers and Enhance Diversity</a> – <a href="https://www.ohsu.edu/school-of-medicine/medical-informatics-and-clinical-epidemiology/college-undergraduate-biomedical">Attracting Talented and Diverse Students to Biomedical Informatics and Data Science Careers Through Short-Term Study at OHSU</a>, 2022-2027 (Grant R25LM014207). A new NLM initiative to enhance diversity and inclusion in informatics by establishing summer internship programs.</li><li>NIH <a href="https://bridge2ai.org/">Bridge2AI Program</a> (Dorr, OHSU PI) – <a href="https://bridge2ai.org/voice/">Voice as a Biomarker of Health</a>, NIH Bridge2AI, 2022-2027 (Award OT2OD032720). An NIH initiative to build high-fidelity, ethically-sourced data sets to advance AI research.</li></ul><p></p><p>As can be seen, I have plenty to keep me busy for some time to come. I plan to continue my work on all of above while supporting the new leadership of the department as it evolves in novel and interesting directions.</p>William Hersh, MDhttp://www.blogger.com/profile/08252786698451811645noreply@blogger.com0tag:blogger.com,1999:blog-1801786186944630495.post-27946012510391569552023-05-22T07:18:00.001-07:002023-05-22T14:28:16.222-07:00It's All Informatics to Me: Data Analytics, Data Science, Digital Health, AI, and Whatever is Next<p>In the last decade or so, a number of new names have emerged for the kinds of activities historically studied, disseminated, and taught by those who work in <i>biomedical and health informatics</i>. Each of these areas has emerged as a "hot topic" in biomedicine and healthcare, with resulting academic funding opportunities, new courses or degree programs, and even academic centers or institutes bearing their names.</p><p>I suppose I have some skin in this game because I have spent my life's work among those developing the field that I consider to go by the over-arching term of this all, <a href="http://informatics.health">biomedical and health informatics</a>. My concern for the new areas and their names has been their often ignoring the historical and current work of informatics, and with it the decades of research, development, implementation, and evaluation of methods and tools that aim to improve health and healthcare through the use of data and information.</p><p>Another ramification of this proliferation of terms is that those not directly working in the field, who may include the leaders of academic and other institutions, may not appreciate the historic role of informatics. Thus I write this post to provide my elaboration of the problem.</p><p>One early new term that started this trend about a decade ago was <i>data analytics</i>. As the growth of large amounts of digital data started with the growth of the Internet, the large-scale implementation of electronic health records, and the emergence of wearable and mobile devices, there were new sources of data that could be analyzed for interesting discoveries. From this started the proliferation of academic courses, certificate/degree programs, and centers/institutes devoted to healthcare (and related areas) data analytics.</p><p>With the proliferation of machine learning, deep learning, and other methods emerged the new discipline of <i>data science</i>. Again, this was followed by the proliferation of academic courses, certificate/degree programs, and centers/institutes focused on data science. One thoughtful perspective of the relationship between informatics and data science was a paper by <a href="https://doi.org/10.1093/jamiaopen/ooy032">Payne et al. in JAMIA Open in 2018</a>, which showed there was not only substantial overlap but also aspects of informatics that go beyond data science. Infomatics includes implementation, evaluation, human-computer interaction, and workflow, to name a few. To use the language of the paper, these were the "efferent" processes of applying knowledge discovered from the "afferent" processes of data science (and informatics) to the real world.</p><p>In the meantime, especially with the growth of telehealth during the COVID-19 pandemic, there emerged a new term, <i>digital health</i>. Similar to data science, there was substantial overlap between digital health and informatics, with that overlap focused on many on the efferent processes of Payne et al. However, for many, the work of digital health is really no different than what informatics has historically done and continues to do.</p><p>And now there is the new term on the block, which is actually not a new term at all but a re-emergence of an area that was the focus of informatics a half-century ago. This term is of course <i>artificial intelligence</i> (AI). Clearly the advances of modern AI are very real and important. The new AI will likely have profound impacts on biomedicine and health, although we also <a href="https://informaticsprofessor.blogspot.com/2021/07/translational-artificial-intelligence.html">need to see real-world implementation and evaluation to know what truly works</a>.</p><p>My goal here is not to discount any of the important advances that have emerged under the guise of data analytics, data science, digital health, or AI. But rather, to express my concern that plenty of people tout these new disciplines sometimes with ignorance of the historic role that informatics has played in them all. I understand that sometimes funders, institutional leaders, and others want to put their resources into things that are "new." But I also am disappointed when these new areas re-invent the methods and tools of informatics.</p><p>The best of informatics imposes a rigor of thinking that these newer disciplines not always apply. Informatics is driven by problems in the domains of health, healthcare, public health, and clinical research. Health and clinical realism is imposed by informatics on the systems as they develop, implemented, and evaluate. I hope that these new areas and what emerges next will not ignore the lessons learned by informatics and incorporate them into their work.</p>William Hersh, MDhttp://www.blogger.com/profile/08252786698451811645noreply@blogger.com0tag:blogger.com,1999:blog-1801786186944630495.post-79594535892440671202023-04-11T15:10:00.003-07:002023-04-11T15:23:10.458-07:00Learning Data Science and Machine Learning with Orange<p>I have written over the years that understanding data science (DS), machine learning (ML), artificial intelligence (AI), and related topics is essential for anyone who works in a profession where tools using them might be employed. This is certainly the case in biomedicine, both in clinical practice and research. In the same sense that all physicians should have basic knowledge of pharmacology, radiology, and other topics, even if they are not experts in the science and methods of those diverse disciplines, all in healthcare should be ready for ML and AI tools coming to their work.</p><p>Of course, learning to apply ML and AI historically has required a background in math and computer programming that most in healthcare do not have. Fortunately, there has been emergence in recent years of a number of "no-code" DS and ML tools. In some ways they are analogous to statistical packages, which allow anyone who knows how to use them load data and run statistical tests. Of course, that does not mean that the operator of statistical software will apply or interpret the results of their analyses correctly, which is especially important for large or complex data sets where appropriate statistical expertise is essential.</p><p>One no-code ML tool that I find particularly easy to understand and use is <a href="https://orangedatamining.com/">Orange Data Mining</a>. Orange was developed by the University of Ljubljana in Slovenia and is an open-source project with a large community supporting it. It runs on Mac and Windows machines, for which binary packages can easily be downloaded and installed. Orange is programmed in Python and, as such, can be extended with Python code. It comes with sample data sets and allows easy loading of new ones. The original focus of Orange was for structured data and machine learning models for operating on them, including the main ML tasks of classification (e.g., logistic regression, kNN, and neural networks) and regression (e.g., linear regression, random forest, and neural networks). There are also methods for visualizing data and evaluating models. The latter includes techniques such as splitting training and test data and cross-fold evaluations are easy to configure. There are also various methods for imputation of missing data. In addition, modules have been developed for image analysis and text mining.</p><p>Orange is used by dragging widgets that represent data sets, visualization methods, models, and more on to a canvas called a workflow. Data flows between the widgets by connecting them via simple dragging between widgets. Once one becomes facile in dragging and connecting widgets, workflows can be built up quickly and easily. They can also be saved and reloaded later.</p><p>One can start with one of the built-in data sets or use the File widget to load one's own data. Most common rectangular data formats are accepted, such as CSV, tab-delimited, and Excel files. Included among the sample data sets is one for heart disease. The data itself is old (Detrano, R., Janosi, A., Steinbrunn, W., Pfisterer, M., Schmid, J., Sandhu, S., Guppy, K., Lee, S., & Froelicher, V. <a href="https://doi.org/10.1016/0002-9149(89)90524-9">International application of a new probability algorithm for the diagnosis of coronary artery disease</a>. <i>American Journal of Cardiology</i>, 1989, 64, 304-310.) and small (303 records with 13 variables), but the data set offers a good way to get started with Orange using clinical data. The outcome variable (called the target variable in Orange) is coronary artery diameter narrowing and the predictor variables include gender, chest pain, serum cholesterol, systolic blood pressure, and more.</p><p>The screenshot below shows a workflow I built using the heart disease data set. As in most data science analyses, one can begin with methods to explore and visualize the data. I connected a number of widgets that allow the data to be viewed in a data table and visualized in the form of feature statistics (also shown below), distributions, and box plots. I also created some models using the Test and Score widget connected to a number of different models appropriate for classification, in this case logistic regression, naive Bayes, and a neural network. One can view the classification accuracy, F1 score, and other evaluation variables (also shown below) and connect the results to widgets that compare ROC curves and generate confusion matrices for each model. (Perhaps not surprisingly, logistic regression works best!) Likewise, I created some models for regression with ST-segment elevation as the outcome variable, using linear regression, random forest, and a neural network, which can also be seen on the workflow.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg4edbUN01x7WX8XDd-SRgT5RJuR6E1nWV2XwPhZAJVR0ek8m4g0xHhLt6jKHB-pQHHJntbG2-iWTVMJqkTnJwUcGpNjDIJ1NeCoeVxr2e_s1y6T5OeXdwTnV7ec4XHIVPmq-qSQZqn-Qr6SLvjIvQT1bHN6jhXO7lBJbXhs3VBlVZ7XmrxbR5o61FzBg/s3018/heart-ows.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1862" data-original-width="3018" height="197" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg4edbUN01x7WX8XDd-SRgT5RJuR6E1nWV2XwPhZAJVR0ek8m4g0xHhLt6jKHB-pQHHJntbG2-iWTVMJqkTnJwUcGpNjDIJ1NeCoeVxr2e_s1y6T5OeXdwTnV7ec4XHIVPmq-qSQZqn-Qr6SLvjIvQT1bHN6jhXO7lBJbXhs3VBlVZ7XmrxbR5o61FzBg/s320/heart-ows.png" width="320" /></a></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiiOa3Lq7gXpsXDlgNgKXq4xmSDEQ9n5e0lNmkOUhZDs-zkCggP7ElerG5gYKjbL6vZrtPC338ADSjOty3A-rEvTWpUnm5Fr3tojlDK9wnkftgGzI8pQYNP0jAE6eN8M4z8WVkkL10iAzsmK9yiGEVBz9wZLeJNveR27o8hU-baplUC6ycH3C3ppJY87w/s2324/featurestatistics.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1280" data-original-width="2324" height="176" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiiOa3Lq7gXpsXDlgNgKXq4xmSDEQ9n5e0lNmkOUhZDs-zkCggP7ElerG5gYKjbL6vZrtPC338ADSjOty3A-rEvTWpUnm5Fr3tojlDK9wnkftgGzI8pQYNP0jAE6eN8M4z8WVkkL10iAzsmK9yiGEVBz9wZLeJNveR27o8hU-baplUC6ycH3C3ppJY87w/s320/featurestatistics.png" width="320" /></a></div><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhDbuBmBEU3gslHKAAXI0Sf7_LmkvMnqNklJQtgnYNtoHWfS_1_H7R5IIN2i93KUZZCHW5lRMA_s-_eCnOqZE71SRN2yjlmNyLZBoMF_Q1zIk9ZkzuOWPcHPG2fxRPvtcPcHPcQLMYVvGdu3i54RfDEhTaEvF5PcjF6eUJVzlB5QYf1-tH42leBcHaWtw/s1784/test&score.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1348" data-original-width="1784" height="242" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhDbuBmBEU3gslHKAAXI0Sf7_LmkvMnqNklJQtgnYNtoHWfS_1_H7R5IIN2i93KUZZCHW5lRMA_s-_eCnOqZE71SRN2yjlmNyLZBoMF_Q1zIk9ZkzuOWPcHPG2fxRPvtcPcHPcQLMYVvGdu3i54RfDEhTaEvF5PcjF6eUJVzlB5QYf1-tH42leBcHaWtw/s320/test&score.png" width="320" /></a></div>It is fairly easy to get started using Orange. A set of <a href="https://www.youtube.com/@OrangeDataMining">video tutorials on the Orange YouTube channel</a> covers the basics as well as a number of advanced topics. My colleague Dr. Robert Hoyt and his collaborator, Dr. David Pashtikoff, have been developing workshops and other learning materials using Orange, available at a Web site they have developed called, <a href="https://www.nocodedatascience.net/">NoCode Data Science</a>. They plan to publish a textbook, and setting up a login on the site allows access to a number of additional data sets that can be loaded into Orange.<p></p><p>Just as with statistical packages, just because one can load data into Orange and apply ML models does not mean that one is doing so correctly. On the other hand, my simple workflow just scratches the surface of what can be done with Orange, even for this tiny data set.</p><p>There are limitations of Orange as a data mining tool. I do not know how scalable it is as data sets get large (all data is kept in computer RAM), and of course any serious ML analysis requires more than the ability to just plug data into models. Nonetheless, Orange can be a great tool for learning machine learning without getting bogged down in coding, debugging, and the like. I see a great future for it in teaching ML as the reach of AI advances.</p>William Hersh, MDhttp://www.blogger.com/profile/08252786698451811645noreply@blogger.com0tag:blogger.com,1999:blog-1801786186944630495.post-25928685882430573362023-03-17T04:05:00.002-07:002023-03-17T04:05:39.422-07:00COVID-19, Three Years On<p>This month marks three years since the start of lockdowns in the United States for the COVID-19 pandemic. As I imagine is true for most people, few upheavals in my life have had such a profound and sustained impact than three years of the pandemic. The good news is that it seems to be receding to an endemic, and while the SARS-CoV-2 is still a threat to many, life is mostly getting back to some semblance of normal.</p><p>Looking back, the year 2020 began like most others. In January, I traveled to Singapore, as I usually did in that month each year, to culminate an offering of the 10x10 course. Little did I realize that would be my last international trip for a year and a half. My final trip prior to the pandemic was a short jaunt to Salt Lake City to give Biomedical Informatics Grand Rounds at the University of Utah. By then, the people on flights and at airports were thinning out. I had decided right before leaving for Salt Lake City to cancel a planned trip to the annual HIMSS Conference in Orlando the following week. Within a few days, that conference was cancelled. I remember the flight back from Salt Lake City, wondering to myself how long it would be before I got on an airplane again. I never would have believed it would be about a year.</p><p>I still remember the dark, early days of the pandemic. Such a lockdown of society was unlike anything I had ever experienced, and our ignorance of the novel SARS-CoV-2 virus was unsettling. All of a sudden, work profoundly transformed from being based in the office to being at home. Fortunately the presence of video tools such as Zoom and WebEx, along with social media, enabled most of us to stay in touch with friends, family, and colleagues. I initially thought the lockdown would be much like the original SARS epidemic of 2003, lasting a few months. I never would have believed that mostly-remote work would continue to now, three years later.</p><p>After three years, the world is returning to some semblance of normal. In a few months, the COVID-19 emergency in the US will officially end. I am comfortable with this decision, although the lack of a pandemic does not mean that the threat of this virus or new ones that may emerge is not real. Just as we have ongoing risk of major killers such as cancer and heart disease, for which we can impact through public health measures, the risk of COVID-19 will continue, and the end of the public health emergency does not mean "COVID is over."</p><p>I prefer to get my information from medical journals and reputable experts. Recent perspective pieces in <a href="https://www.nejm.org/doi/full/10.1056/NEJMp2213920">New England Journal of Medicine</a> and <a href="https://www.nature.com/articles/d41586-022-00155-x">Nature</a> summarize the situation well. I have been a follower of Dr. Eric Topol's tweeting and now Substack during the pandemic, and a recent <a href="https://erictopol.substack.com/p/a-break-from-covid-waves-and-a-breakthrough">perspective</a> from him is enlightening. And the Washington Post had a good <a href="https://www.washingtonpost.com/health/2023/03/12/doctors-covid-new-normal/">op-ed</a> asking a number of COVID-19 experts how they are adapting to the current situation.</p><p>My view is that now that COVID-19 is receding to endemic status, we can revisit our approaches to the virus. The virus will be with us going forward, and continue to be a major cause of death and other disability. But I am comfortable with moving to an individual risk assessment approach. I still take precautions. While I no longer routinely mask in public, including indoor settings, I always carry a mask and use it sometimes in crowded indoor settings. I always mask when someone else asks me to do so. I am up to date on vaccines, having had five, including the bivalent booster (as well as one mild natural infection in 2022).</p><p>Probably the saddest aspect of the COVID-19 pandemic is the politicization of public health. I get that there are trade-offs in the world between personal liberty and the public good. And I believe an honest re-assessment of what we thought we did right or wrong early in the pandemic is not a bad idea. Clearly we were flying blind in those dark early days. While I do not believe that the early lock-downs and mask requirements were unreasonable at the time, we could have an honest re-assessment now of what is the best approach. By the time token, we can never forget that over a million US lives have been lost and many more disrupted by the virus. Those who pushed unproven treatments and approaches like "let er rip" herd immunity strategies likewise need to called out for their wrong ways.</p><p>The threat of COVID-19 is still very real. It remains the third-leading cause of death in the US. It creates a substantial risk for those who are vulnerable, such as those who are immunocompromised or elderly. Data from the CDC and around the world show that while the vaccine does not eradicate the virus or completely prevent its transmission, it does reduce the risk of death and hospitalization, especially for those at most risk.</p><p>By the same token, I feel comfortable going mostly maskless, including on airplanes, going to the gym, and at conferences and other public venues. I know that I could again get infected by SARS-CoV-2, but I believe my risk of serious illness is small. I like to think of myself as a COVID "tweener," taking the virus seriously but comfortable returning to relatively normal living. I am ready to return to more aggressive protection should pandemic status recur.</p>William Hersh, MDhttp://www.blogger.com/profile/08252786698451811645noreply@blogger.com0tag:blogger.com,1999:blog-1801786186944630495.post-57921859142925417682023-02-06T05:30:00.004-08:002023-02-06T05:33:00.220-08:00Who Should Program?<p>Like many who work with computers and other information technology, I enjoy programming. I got my start in programming when I was first exposed to computers in high school in the 1970s, when almost all you could do with computers required programming. Like many at the time, the first programming language I learned was BASIC. I subsequently cycled through many of the common programming languages of the 1970s and 1980s, including PL/1, Pascal, and C (although, for whatever reason, never learned FORTRAN or COBOL). I even dabbled some in assembly language during my medical informatics postdoc fellowship in the late 1980s. Once I started my biomedical informatics academic faculty position in 1990, I began to follow the lead on programming languages from students and other trainees, through Perl, Python, R, and others. </p><p>But as I ascended the ranks of academic leadership, my time for programming diminished. I still have periodic bursts of learning and programming now, but do not have the sustained time available to be productive with them. In some ways, programming is akin to the practice of medicine for clinicians who work in academic informatics - one may be able to keep up with the knowledge of clinical medicine, or even master access to knowledge through modern tools - but still the practice of medicine requires a critical mass of time commitment for one to remain an effective clinician. I learned this in the early 2000s when my informatics work pretty much crowded out my ability to effectively provide patient care, especially trying to practice as a primary care internist.</p><p>Interestingly, I sometimes I get into discussions with physicians who are training in informatics and desire to commit a great deal of time and effort to programming. Some of them can pull it off, but as I note to many, they are probably too expensive to be programmers (even as salaries for high-end programmers are reaching the lower strata of physician salaries). Furthermore, physicians have knowledge and perspectives that may be most useful for tasks other than programming, such as conceptual thinking and clinical leadership.</p><p>It is actually questionable whether to leave programming to those who are academics, including biomedical researchers. One recent analysis looked at a repository of replication data sets and code published in the Harvard Dataverse Repository between 2010-2020 (1). It found that about three-quarters of R programs failed to execute correctly, although there was some improvement when code-cleaning was applied.</p><p>Another challenge is that code sometimes is maintained by a small number or even just one person, the latter in the case of a particle physicist who was the sole maintainer of a 1980s program and has now retired (2). Some have expressed concern for continued progress in the field. Previous work found similar problems in functional MRI (3) and computational chemistry (4) analysis. And many geneticists have had to deal with idiosyncrasies such as the propensity of Microsoft Excel to change gene names to dates (e.g., MARCH1 to 1-Mar) (5).</p><p>By the way, I am in no way advocating that modern-day higher-education students, especially in science fields, <u>not</u> learn computer programming as part of their education. Programming is an important skill and discipline, facilitating critical and logical thinking. As computers play an important role in so many knowledge-based disciplines in modern times, it is critical to de-mystify them and allow them to be used most effectively in all scientific disciplines.</p><p>Another important development over the years has been the emergence of tools that allow management and manipulation of data with minimal amounts of programming. Probably the earliest example of this was the spreadsheet. Also of note are statistical software packages that allow one to run a myriad of statistical tests with little or no programming. Of course, in both of these examples, one must still understand their data and appropriate inferences drawn from it. Another more recent example of this is packages such as <a href="https://orangedatamining.com/">Orange</a>, which provide a visual programming interface to many data science and machine learning methods. I will have more to say about Orange in a future post.</p><p>We can also gain insight from the recent development of <a href="https://informaticsprofessor.blogspot.com/2023/01/chatgpt-confabulates-and-is.html">ChatGPT</a>. One prominent early use has been asking it to write code for specific functions in different languages. While ChatGPT does produce code that looks like what is being asked for, programming support sites like Stack Overflow have banned postings of such code because although they look correct, there are often subtle errors or other problems (6).</p><p>It is clear that there will always be a role for learning and exploring with computer programming. But given the complexity of modern software, and the need for programs to be robust, correct, and providing security, those who enjoy programming should be realistic about the commitment required to carry it out at "industrial-strength" levels.</p><p>References</p><p>1. Trisovic, A., Lau, M.K., Pasquier, T., Crosas, M., 2022. A large-scale study on research code quality and execution. Sci Data 9, 60. <a href="https://doi.org/10.1038/s41597-022-01143-6">https://doi.org/10.1038/s41597-022-01143-6</a></p><p>2. von Hippel, M., 2022. Crucial Computer Program for Particle Physics at Risk of Obsolescence. Quanta Magazine. <a href="https://www.quantamagazine.org/crucial-computer-program-for-particle-physics-at-risk-of-obsolescence-20221201/">https://www.quantamagazine.org/crucial-computer-program-for-particle-physics-at-risk-of-obsolescence-20221201/</a></p><p>3. Hirschler, B., 2014. Exclusive: Software issue casts doubt over data used to approve some drugs. Reuters. <a href="http://finance.yahoo.com/news/exclusive-software-issue-casts-doubt-over-data-used-121215317--finance.html">http://finance.yahoo.com/news/exclusive-software-issue-casts-doubt-over-data-used-121215317--finance.html</a></p><p>4. Eklund, A., Nichols, T.E., Knutsson, H., 2016. Cluster failure: Why fMRI inferences for spatial extent have inflated false-positive rates. Proc Natl Acad Sci U S A 113, 7900–7905. <a href="https://doi.org/10.1073/pnas.1602413113">https://doi.org/10.1073/pnas.1602413113</a></p><p>5. Vincent, J., 2020. Scientists rename human genes to stop Microsoft Excel from misreading them as dates [WWW Document]. The Verge. <a href="https://www.theverge.com/2020/8/6/21355674/human-genes-rename-microsoft-excel-misreading-dates">https://www.theverge.com/2020/8/6/21355674/human-genes-rename-microsoft-excel-misreading-dates</a></p><p>6. Vincent, J., 2022. AI-generated answers temporarily banned on coding Q&A site Stack Overflow. The Verge. <a href="https://www.theverge.com/2022/12/5/23493932/chatgpt-ai-generated-answers-temporarily-banned-stack-overflow-llms-dangers">https://www.theverge.com/2022/12/5/23493932/chatgpt-ai-generated-answers-temporarily-banned-stack-overflow-llms-dangers</a></p>William Hersh, MDhttp://www.blogger.com/profile/08252786698451811645noreply@blogger.com0tag:blogger.com,1999:blog-1801786186944630495.post-43349681715890680752023-01-25T06:20:00.000-08:002023-01-25T06:20:25.767-08:00ChatGPT Confabulates and is Unprofessional<p>Since <a href="https://informaticsprofessor.blogspot.com/2022/12/chatgpt-postscript.html">my own explorations with ChatGPT</a> in informatics knowledge last month, others using the system have racked up some impressive results. Most pertinent to biomedical informatics is ChatGPT's <a href="https://doi.org/10.1101/2022.12.19.22283643">passing grades on medical student licensing exams</a>. It has also performed well on <a href="https://mackinstitute.wharton.upenn.edu/2023/would-chat-gpt3-get-a-wharton-mba-new-white-paper-by-christian-terwiesch/">Wharton MBA</a> and <a href="https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4314839">multi-state bar exams</a>.</p><p>Does this mean that machine intelligence has equaled human professionals? My subsequent encounters with ChatGPT lead me to maintain my skepticism. Not that ChatGPT is not a tremendous feat, but it is not (yet anyways) capable of performing as a physician, lawyer, or MBA.</p><p>This conclusion is evidenced from another recent interaction I had with ChatGPT, the transcript of which is at the bottom of this post. This interaction mirrors some characteristics that many others note. One property that many find is ChatGPT's tendency to confabulate. As seen in my latest interaction, it states things that are simply false. Not that humans do not do this, but no one would rely on a physician, lawyer, or MBA known to flagrantly lie. As such, ChatGPT fails at one core task of being a competent professional in a given field, where there is an ethical responsibility to recognize and state when you do not know something. The best physician is one who knows the limits of his or her knowledge and skills. (Of course, ChatGPT lacks many other capabilities of a physician, from technical skill to empathy.)</p><p>This leads me to a thought experiment, perhaps somewhat biased by the fact I am an information retrieval (IR) researcher. If you were confronted with a highly challenging intellectual task, which would you use first, Google (or another search system, maybe PubMed for medicine) or ChatGPT? In my mind, an IR system is much more appealing, giving what IR systems provide by definition, which is pointers to information that may answer questions. Google is far from a perfect search engine, but when addressing a complicated intellectual task, I would much rather take my chances with an IR system output with primary sources listed and accessible than the narrative without citations that comes out of ChatGPT.</p><p>I have no doubt that systems like ChatGPT will continue to advance, and that human knowledge workers will find ways to use them beneficially in their work. But I cannot imagine there not being a human in the loop somewhere, especially in medicine when you might need the skills of a surgeon, not to mention a physician with empathy and grace. I am also reminded of the quote from my colleague, radiologist <a href="https://pubs.rsna.org/doi/10.1148/ryai.2019190058">Curtis Langlotz, probably speaking for all of medicine, who said</a>, "Will AI replace radiologists? is the wrong question. The right answer is: Radiologists who use AI will replace radiologists who don’t."</p><p>Here is the transcript from my most recent interaction with ChatGPT, with my <i>annotations in italics</i>. I would also note, perhaps in a blow to my own ego, that ChatGPT's answer does not include any of my own IR writings, such as the <a href="http://irbook.info/">recent edition of my biomedical IR textbook</a>, which arguably are important to this field.</p><p>I entered to Chat GPT, What are some good readings for biomedical information retrieval?</p><p>There are many resources available for learning about biomedical information retrieval, including books, articles, and online courses. Here are a few suggestions:</p><p>"Introduction to Biomedical Information Retrieval" by C. J. van Rijsbergen is a classic textbook that provides a comprehensive introduction to the field.</p><p><i>Keith van Rijsbergen is a retired computer science professor and IR researcher who has written several books but not one called "Introduction to Biomedical Information Retrieval."</i></p><p>"Information Retrieval in Biomedicine" by Peter Willett is another comprehensive textbook that covers the basics of biomedical information retrieval as well as advanced topics.</p><p><i>Peter Willett is a retired computer science professor and IR researcher who has written several books but not one called "Information Retrieval in Biomedicine."</i></p><p>"Medical Information Retrieval" by J. V. McInerney is a book that provides an overview of the field and covers various topics such as search engines, natural language processing, and data mining.</p><p><i>I am not able to find any book titled "Medical Information Retrieval" or anyone in the IR or informatics field named J. V. McInerney.</i></p><p>"Biomedical Information Retrieval" by J. S. Ogasawara and M. R. Al-Rawi is a book that covers various aspects of biomedical information retrieval, including text mining, ontologies, and evaluation.</p><p><i>I am not able to find any book titled "Biomedical Information Retrieval" or anyone in the IR or informatics fields named J. S. Ogasawara or M. R. Al-Rawi.</i></p><p>"Biomedical Text Mining" by Haifeng Li and Linyun Fu is a book that covers the use of text mining techniques for extracting and analyzing information from biomedical literature.</p><p><i>There is a book named "Biomedical Text Mining" but it does have authors Haifeng Li or Linyun Fu, neither of whom work in IR or informatics.</i></p><p>The Journal of Biomedical Informatics is a peer-reviewed journal that publishes research on various aspects of biomedical information retrieval.</p><p><i>This is true!</i></p><p>The International Conference on Medical Information Retrieval is an annual conference that brings together researchers and practitioners in the field to present and discuss their work.</p><p><i>I am not aware of any such conference by this name.</i></p><p>I hope these suggestions are helpful! Let me know if you have any other questions.</p><p><i>I will!</i></p>William Hersh, MDhttp://www.blogger.com/profile/08252786698451811645noreply@blogger.com0tag:blogger.com,1999:blog-1801786186944630495.post-1795585887072495822022-12-31T04:05:00.001-08:002022-12-31T04:10:58.367-08:00Annual Reflections at the End of 2022<p>I am keeping my tradition in this blog this year of ending the calendar year with a retrospective on the past year and some musings looking forward.</p><p>Clearly our lives were still dominated in 2022 by the COVID-19 pandemic. While a fair amount of living has returned to relative normal, there are some changes that will likely never fully reverse. Certainly a major one for myself and my immediate co-workers is the likely continuation of mostly remote work.</p><p>Looking back, my last annual year-end post from the "before times" was in 2019. It was an ordinary post, discussing the history of the blog starting from the times of the passage of the HITECH Act, along with the continued growth of my teaching and use of Twitter. I noted over 2600 people having completing the 10x10 course and my surpassing 2600 followers on Twitter. Now at the end of 2022, 10x10 has exceeded 3100 people completing the course and my Twitter followers number over 3500. Of course, with the recent acquisition of Twitter by Elon Musk, the future of this platform is uncertain. That is unfortunate, since I find activities like #ScienceTwitter and #MedTwitter very helpful for keeping up with new advances, some of which informs my teaching.</p><p>The day of that 2019 year-end post was the day that the first case of SARS-CoV2 infection was documented in China. Within a month cases started appearing in the US, and by March, the <a href="https://informaticsprofessor.blogspot.com/2020/03/sars-cov-2-course-ahead.html">general US lockdown began</a>, with the transition to remote work. While we have made great progress in treatments and vaccinations for COVID-19, my work is still mostly remote. In reality, this is an extension of pre-pandemic times, when I even noted then that <a href="https://informaticsprofessor.blogspot.com/2019/08/living-asynchronous-life.html">my work was mostly asynchronous</a>. And it is not hard to see how it will likely continue mostly that way, as there are plenty of advantages to my office commute being the walk down the stairs to my home office in my basement. I also enjoy being able to do my daily exercise (running or gym) in the late morning, followed by a healthy lunch, and then some quiet meditation after lunch before getting back to work. In the early days of pandemic, the Internet was highly overtaxed in my neighborhood of many professionals working from home, but the infrastructure has been built up such that my current home Internet is well up to the task.</p><p>Back to this year, it was a good year for my professional and personal life. I made the <a href="https://informaticsprofessor.blogspot.com/2022/09/a-new-pathway-for-my-career-in.html">decision to hand off the leadership of the department I have served as the one and only Chair</a>. The Dean of the OHSU School of Medicine will make an announcement of the appointment of a new Interim Chair next week. As noted in my <a href="https://informaticsprofessor.blogspot.com/2022/12/kudos-for-informatics-professor-2022.html">kudos posting from last week</a>, my research and teaching work was highly productive in 2022, and the activities and grant funding will keep it going for at least several more years. I aim to keep working focused on research, teaching, mentoring, and writing, and have no plans to retire any time soon. I will also maintain this blog, which with this posting now has 375 postings.</p><p>As for the future, COVID-19 will continue to impact all of what we do. But there is less fear to have now, as we understand the virus and the disease it causes much better, and we have means to blunt the worst of its effects. Many quip that I have an optimistic view of the world and future, and I am certain this will continue in 2023 including and beyond COVID-19.</p>William Hersh, MDhttp://www.blogger.com/profile/08252786698451811645noreply@blogger.com0tag:blogger.com,1999:blog-1801786186944630495.post-12250536577091849822022-12-19T04:33:00.000-08:002022-12-19T04:33:26.465-08:00Kudos for the Informatics Professor - 2022 Update<p>The year 2022 was a highly productive one for the Informatics Professor, and I am pleased to provide my periodic update of my recent accomplishments. This year had another big event in <a href="https://informaticsprofessor.blogspot.com/2022/09/a-new-pathway-for-my-career-in.html">my decision to step down as the one and only Chair</a> of the <a href="http://www.ohsu.edu/informatics">Oregon Health & Science University (OHSU) Department of Medical Informatics & Clinical Epidemiology</a>. More will be posted about that when my successor is named. Meanwhile, here are my accomplishments for 2022.</p><p>I received two prominent awards this year. One was from the <a href="https://amia.org/">American Medical Informatics Association (AMIA)</a>, my main professional association, for the <a href="https://amia.org/about-amia/amia-awards/signature-awards/william-w-stead-award-thought-leadership-informatics">William Stead Award for Thought Leadership in Informatics</a>. This award "acknowledges people who have influenced our thinking about informatics, especially improving health and health care in ways that are visionary and transformative." This year I was also elected to the second class of the <a href="https://sigir.org/awards/sigir-academy/">ACM SIGIR Academy</a>, which was established to "honor and recognize individuals who have made significant, cumulative contributions to the development of the field of information retrieval (IR)." I am thrilled to be honored by the two fields of which I work at the intersection between them.</p><p>This year I completed my term as the Second President of the <a href="https://imia-medinfo.org/wp/iahsi/">International Academy of Health Sciences Informatics (IAHSI)</a>. I will continue for another two years as Past President.</p><p>In 2022, I was awarded several new grants, which will help sustain the research and teaching that I plan to continue as I reduce my administrative and leadership roles. Probably the most important of these was the <a href="https://blogs.ohsu.edu/health-data/2022/05/04/national-library-of-medicine-t15-training-grant-in-biomedical-informatics-and-data-science-renewed/">OHSU Training Grant in Biomedical Informatics and Data Science</a>, which was renewed for another five year cycle through 2027. This <a href="https://www.nlm.nih.gov/ep/GrantTrainInstitute.html">training grant from the National Library of Medicine (NLM)</a> of the <a href="https://www.nih.gov/">National Institutes of Health (NIH)</a> funds predoctoral (PhD) and postdoctoral trainees in the <a href="https://www.ohsu.edu/school-of-medicine/medical-informatics-and-clinical-epidemiology/prospective-students">OHSU Biomedical Informatics Graduate Program</a>. This grant has been funded continuously at OHSU since 1992. This renewal marks the seventh consecutive five-year cycle of funding for OHSU. Its first 30 years make it the second-longest-running training grant at OHSU. The grant's total funding over that time of $21,912,538 make it the largest training grant ever at OHSU. I myself am a product of this training grant, completing my postdoctoral fellowship at Harvard University from 1987-1990.</p><p>Another grant from the NLM for which I am PI is an R25 grant to fund the <a href="https://www.ohsu.edu/school-of-medicine/medical-informatics-and-clinical-epidemiology/college-undergraduate-biomedical">OHSU Biomedical Informatics and Data Science College Undergraduate Summer Internship Program</a> for five years. The grant is among <a href="https://www.nlm.nih.gov/news/NLM_Invests_8_Million_To_Develop_Diverse_Biomedical_Informatics_Data_Science_Training_Programs.html">12 awards issued by the National Library of Medicine to prepare historically underrepresented students for biomedical informatics and data science graduate studies and research careers</a>. The goal of the our program is to provide a rich, full-time experience to students to graduate study and careers in biomedical informatics and data science. In addition, the program aims to motivate students to pursue doctoral training in biomedical informatics and data science that leads to research careers in academia, government, healthcare, and industry. The funding from the R25 grant will be used to support summer experiences for students with backgrounds that have been historically underrepresented in biomedical informatics and data science.</p><p>A final new grant for 2022 was from the new <a href="https://commonfund.nih.gov/bridge2ai">NIH Bridge2AI Initiative</a>. I am among a group of faculty from OHSU and Washington University who will be serving as the <a href="https://bridge2ai.org/skills-and-workforce-development-core/">Skills and Workforce Development Module</a> for one of the four funded data-acquisition projects. The project we are part of will focus on creating data sets that aim to use <a href="https://reporter.nih.gov/project-details/10473236">voice as a biomarker of health</a>.</p><p>I also had a good year of publications in 2022, not the least of which was the release of the 8th edition of <a href="http://www.informaticsbook.info">Health Informatics: Practical Guide</a>, in which I serve as Editor and have authored about a third of the chapters. My journal publications for 2022 included the following:</p><p></p><ul style="text-align: left;"><li>Hersh WR, Cohen AM, Nguyen MM, Bensching KL, Deloughery TG, <a href="https://academic.oup.com/jamiaopen/article/5/2/ooac053/6621904">Clinical study applying machine learning to detect a rare disease: results and lessons learned</a>, <i>JAMIA Open</i>, 2022, 5: ooac053.</li><li>Hersh WR, <a href="https://www.thieme-connect.com/products/ejournals/abstract/10.1055/s-0042-1745722">The clinical informatics practice pathway should be maintained for now but transformed into an alternative to in-place fellowships</a>, <i>Applied Clinical Informatics</i>, 2022, 13: 398-399. PMC8942720.</li><li>Hersh WR, <a href="https://ebooks.iospress.nl/doi/10.3233/SHTI220944">Competencies and curricula across the spectrum of learners for biomedical and health informatics</a>, <i>Achievements, Milestones and Challenges in Biomedical and Health Informatics</i>, 2022, 93-107.</li><li>Baker CK, Maniam N, Schnapp BH, Genes N, Nielson JA, Mohan V, Hersh W, Slovis, BH, <a href="https://jetem.org/ci_curriculum/">A model curriculum for an emergency medicine residency rotation in clinical informatics</a>, <i>Journal of Education and Teaching in Emergency Medicine</i>, 2022, 7(4)C1-C50.</li><li>Bichel-Findlay J, Koch S, Mantas J, Abdul SS, Al-Shorbaji N, Ammenwerth E, Baum A, Borycki EM, Demiris G, Hasman A, Hersh W, Hovenga E, Huebner UH, Huesing ES, Kushniruk A, Lee KH, Lehmann CU, Lillehaug SI, Marin HF, Marschollek M, Martin-Sanchez F, Merolli M, Nishimwe A, Saranto K, Sent D, Shachak A, Udayasankaran JG, Were MC, Wright G, <a href="https://www.sciencedirect.com/science/article/pii/S1386505622002222">Recommendations of the International Medical Informatics Association (IMIA) on education in biomedical and health informatics: second revision</a>, <i>International Journal of Medical Informatics</i>, 2023, 170: 104908.</li></ul><p></p><p>In 2022, I was also a guest on a couple of podcasts. One was to provide an introduction to an <a href="https://amia.org/news-publications/podcasts/for-your-informatics/your-informatics-episode-26-gender-pay-gap-medicine">episode</a> on of the <a href="https://amia.org/news-publications/podcasts/for-your-informatics">For Your Informatics</a> podcast on the gender pay gap among physicians, including those in informatics. The other was to be interviewed for the <a href="https://podcasts.apple.com/us/podcast/exploring-healthcare-interoperability/id1570910011">IAHSI podcast on Exploring Healthcare Interoperability</a> about my work in <a href="https://podcasts.apple.com/us/podcast/episode-3-healthcare-informatics-and-interoperability/id1570910011?i=1000558774692">teaching about healthcare data standards and interoperability</a>.</p><p>I also gave a number of invited talks in 2022:</p><p></p><ul style="text-align: left;"><li><i>Biomedical and Health Informatics: Impact, Challenges, and Opportunities</i>, Nevada Institute of Personalized Medicine, University of Nevada Las Vegas (virtual), January 27, 2022 (<a href="https://dmice.ohsu.edu/hersh/unlv-informatics.pdf">slides and references</a>)</li><li><i>Competencies and Curricula Across the Spectrum of Learners for Health Informatics</i>, Keynote Talk, IEEE International Conference on Healthcare Informatics, First International Workshop on Health Informatics Education, Rochester, MN, June 11, 2022 (<a href="https://dmice.ohsu.edu/hersh/ieeeichi-22-edu.pdf">slides and references</a>)</li><li><i>Translational Artificial Intelligence (AI): The Need to Translate from Basic Science to Clinical Value</i>, University of Alabama Birmingham Informatics Institute (virtual), September 2, 2022 (<a href="https://dmice.ohsu.edu/hersh/uab-22-transai.pdf">slides and references</a> and <a href="https://us02web.zoom.us/rec/play/1gVU-rseJHVqyja09AXrPa6J7p68kg73Ox1CJsn3H6oa2pWZ2WI0truhFBxZItz9zhRDCsUR0VAO7LSR.B4kDGugL4DJr3JjZ?autoplay=true&startTime=1662130940000">video</a>)</li><li><i>Past, Present, and Future: A Discussion About DMICE Going Forward</i>, OHSU Department of Medical Informatics & Clinical Epidemiology, October 6, 2022 (<a href="https://dmice.ohsu.edu/hersh/PastPresentFuture.pdf">slides</a> and <a href="https://www.youtube.com/watch?v=L2oE7l-xT4E&t=2902s">video</a>)</li><li><i>Applying Information Retrieval to the Electronic Health Record for Cohort Discovery and Rare Disease Detection</i>, Brown University Center for Biomedical Informatics, Providence, RI, October 13, 2022 (<a href="https://dmice.ohsu.edu/hersh/brown-22-ehrir.pdf">slides and references</a>)</li><li><i>Competencies and Curricula Across the Spectrum of Learners for Biomedical and Health Informatics</i>, International Symposium on Achievements, Milestones and Challenges in Biomedical and Health Informatics, Athens, Greece, October 29, 2002 (<a href="https://dmice.ohsu.edu/hersh/symposio-22-slides.pdf">slides</a>)</li></ul><p></p><p>In 2022 I also delivered my annual lecture to first-year OHSU medical students, <a href="https://dmice.ohsu.edu/hersh/information-different-22.pdf">Information is Different Now That You're a Doctor</a>, on September 2, 2022. I also gave this talk to first-year medical students at Nova Southeastern University (virtual) on June 16, 2022.</p><p>I additionally attended my 37th consecutive AMIA Annual Symposium, where I participated in three activities:</p><p></p><ul style="text-align: left;"><li>Panel: Beyond Wrangling and Modeling: Data Science and Machine Learning Competencies and Curricula for The Rest of Us</li><li>Job Talk Panel for the AMIA Clinical Informatics Fellows (ACIF)</li><li>Debate: Extension of the Practice Pathway for ABMS Clinical Informatics Board Certification for Physicians in the United States</li></ul><p></p><p>This year I also served as Co-Chair of the Scientific Program Committee of the conference hosted by Brown University, <a href="https://cme-learning.brown.edu/PromoteHealing#group-tabs-node-course-default1">The Patient and The Practitioner in the Age of Technology: Promoting Healing Relationships</a>.</p><p>In 2022, I also maintained my various scientific citation rankings. In the list of <a href="https://research.com/scientists-rankings/computer-science/us">Best Computer Science Scientists</a>, I now rank 694th in the US and 1183rd in the world based on a discipline-specific H-index they calculate. I also maintain a presence in the latest iteration of a database of the <a href="https://elsevier.digitalcommonsdata.com/v1/datasets/compare/btchxktzyw/4/5">top 100,000 cited scientists in the world for 22 scientific fields and 176 subfields</a>, where I rank 22,553rd based on a composite formula of citation and publishing variables. In the medical informatics subfield, I rank 13th. Of the 290 individuals affiliated with OHSU, I rank 50th. I also maintain my presence on ranking lists based on the Google Scholar h-index for <a href="http://allisonbmccoy.github.io/scholar-scraper/index-bmi.html">biomedical informatics</a> and <a href="https://lintool.github.io/bigcows/index-ir.html">information retrieval</a>.</p><p>Finally, this blog is mentioned in various lists of best blogs, including the following:</p><p></p><ul style="text-align: left;"><li><a href="https://blog.feedspot.com/bioinformatics_blogs/">70 Best Bioinformatics Blogs and Websites</a></li><li><a href="https://blog.feedspot.com/health_informatics_blogs/">35 Best Health Informatics Blogs and Websites</a></li></ul>William Hersh, MDhttp://www.blogger.com/profile/08252786698451811645noreply@blogger.com0tag:blogger.com,1999:blog-1801786186944630495.post-30669915485591918002022-12-17T08:11:00.007-08:002022-12-21T15:30:51.353-08:00ChatGPT Postscript<p>It has been interesting to read all of the uses that colleagues of mine and others have employed to put ChatGPT to the test, in addition to my tests of <a href="https://informaticsprofessor.blogspot.com/2022/12/initial-excursions-with-chatgpt.html">learning objectives, a biography</a>, and <a href="https://informaticsprofessor.blogspot.com/2022/12/chatgpt-round-two.html">multiple-choice questions</a>. In one email thread, I joked that perhaps ChatGPT is a Rorschach Test of what interests people concerning the use of artificial intelligence.</p><p>I also was pointed to an interesting site that bills itself as a ChatGPT (actually trained on an earlier version of the OpenAI model, GPT-2) <a href="https://huggingface.co/openai-detector/">Output Detector Demo</a>. I pasted my biography from my first post and this system declared the text had a 99.98% chance of being "fake," i.e., from GPT-2. When I paste in in the biographic paragraph from my <a href="http://www.billhersh.info">own Web page</a>, it declares the text to having a 99.97% chance of being real.</p><p>Another interesting reflection is to compare ChatGPT with <a href="http://www.irbook.info">information retrieval (IR, aka search)</a>. Perhaps I am biased as an academic, or someone greatly interested in IR since it is my primary focus of research, but usually when I look for information, I not only want to know the information, but also where it comes from and how trustworthy it is. A big limitation for me of ChatGPT is that it cites no references to back up what it says.</p><p>This gets to another academic concern about ChatGPT, which is how it will impact assessment of learning. Although ChatGPT seems to work best for relatively short passages of text that do not require references, there are fortunately many other ways to assess learning.</p><p>There have also been some good overviews in the news media about ChatGPT, including an <a href="https://spectrum.ieee.org/chatbot-chatgpt-interview">interview of ChatGPT itself</a>. There is also a <a href="https://www.nytimes.com/2022/12/05/technology/chatgpt-ai-twitter.html">nice description from the New York Times</a>.</p>William Hersh, MDhttp://www.blogger.com/profile/08252786698451811645noreply@blogger.com0tag:blogger.com,1999:blog-1801786186944630495.post-7297656282855956732022-12-13T06:06:00.001-08:002022-12-13T06:06:41.486-08:00ChatGPT, Round TwoAfter sending some comments about my <a href="https://informaticsprofessor.blogspot.com/2022/12/initial-excursions-with-chatgpt.html">blog post on my initial excursions with ChatGPT</a> earlier this week, I received a great deal of interesting follow-up. One colleague lamented that we may no longer be able to assess the knowledge of students, especially when teaching in online classes, because of their ability to easily generate answers to questions using systems like ChatGPT, especially if they could add to the text by Googling a few references to add to their answers.<div><br /></div><div>I am not convinced of this, at least not with the use of ChatGPT, whose answers I noted were superficial, not completely correct, and not backed up with citations. Furthermore, asking for short answers to questions is hardly the only way to assess knowledge.</div><div><br /></div><div><div>So to put ChatGPT to the next test, I pasted in, one by one, the 10 multiple-choice questions I give for the same unit of my <a href="https://dmice.ohsu.edu/hersh/10x10.html">introductory informatics course</a> as the learning objectives of my first assessment.</div><div> </div><div>While I have to give ChatGPT some credit for figuring out these were multiple-choice questions, it performed pretty poorly compared to most students in the course. It only answered 5 of the 10 questions correctly, and for two it got wrong, it did not even try to answer the questions, giving the excuse it could not browse the Internet (despite being trained on it!).</div><div> </div><div>Some of the incorrect answers would be embarrassing even for a beginning informatics student to not know, such as which <a href="https://www.nlm.nih.gov/ep/GrantTrainInstitute.html">US government agency funds informatics training</a>, that "meaningful use" was not part of the Affordable Care Act, and that a machine learning algorithm not involving clinicians does not adhere to <a href="https://doi.org/10.1197/jamia.M3092">Friedman's Fundamental Theorem</a>.</div><div> </div><div>The transcript of my interactions is pasted in below. The text entered into ChatGPT did not include the question numbers nor the correct answers indicated by asterisks. The paragraph following each question is how ChatGPT replied. My conclusion is that, for now, those of us who teach informatics in virtual settings do not (yet) have to worry about ChatGPT being used to cheat in our courses.</div></div><div><br /></div><div>ChatGPT Transcript - December 10, 2022</div><div><br /></div><div><div>1. Which part of biomedical and health informatics is most likely to address the critical informatics issues for the practice of a family physician?</div><div>a. Bioinformatics</div><div>*b. Clinical informatics</div><div>c. Clinical research informatics</div><div>d. Imaging informatics</div><div>e. Public health informatics</div><div><br /></div><div>Clinical informatics is most likely to address the critical informatics issues for the practice of a family physician. This subfield of biomedical and health informatics focuses on the use of information technology to support the delivery of healthcare and improve patient outcomes. Clinical informatics encompasses a wide range of topics, including electronic health records, health information exchange, clinical decision support, and population health management. It is concerned with the design, development, and implementation of systems and tools to support the clinical work of healthcare professionals, such as family physicians.</div><div><br /></div><div>2. Before the era of widespread computer use, the typical hospital physician spent what proportion of his or her time in activities away from the patient during office practice?</div><div>*a. 23-39%</div><div>b. 50%</div><div>c. 61-76%</div><div>d. 100%</div><div><br /></div><div>I'm sorry, but I am not able to browse the internet to answer your question as I am a large language model trained by OpenAI. My knowledge is based solely on the text that I have been trained on, which has a cutoff date of 2021. I do not have any additional information beyond what I have been trained on.</div><div><br /></div><div>3. Which entity of the US government funds research and training in biomedical and health informatics?</div><div>a. Agency for Healthcare Research & Quality</div><div>b. Center for Medicare and Medicaid Services</div><div>c. National Committee for Vital & Health Statistics</div><div>*d. National Library of Medicine</div><div><br /></div><div>The National Institutes of Health (NIH) is the primary entity of the US government that funds research and training in biomedical and health informatics. The NIH is a federal agency that is part of the Department of Health and Human Services. It is the largest biomedical research agency in the world and provides funding for a wide range of research and training programs in the field of biomedical and health informatics. Some examples of NIH-funded programs in this area include the National Library of Medicine's Training and Education in Biomedical Informatics program and the Clinical and Translational Science Awards program, which support research and training in clinical informatics.</div><div><br /></div><div>4. Before the emergence of the Affordable Care Act, a number of proposed healthcare reform plans for the US would provide for the establishment of a small number of highly regulated private insurance companies in a given region or state. This type of plan would be described by JR Reid as:</div><div>a. Beveridge</div><div>*b. Bismarck</div><div>c. National health insurance</div><div>d. Out-of-pocket</div><div><br /></div><div>The type of healthcare reform plan described in the question, which involves the establishment of a small number of highly regulated private insurance companies in a given region or state, would be described by JR Reid as a Bismarck-type plan. Bismarck-type plans are named after Otto von Bismarck, the 19th century Prussian statesman who introduced the first national health insurance program in the world. Bismarck-type plans are characterized by a high degree of government control over the healthcare system, with the state playing a central role in regulating the provision of healthcare services and financing. In these plans, private insurance companies are typically required to operate within a regulatory framework set by the government, and may be subject to price controls and other forms of regulation. This type of plan is common in many European countries.</div><div><br /></div><div>5. What proportion of US healthcare spending is devoted to prescription drugs?</div><div>*a. 10%</div><div>b. 20%</div><div>c. 33%</div><div>d. 50%</div><div><br /></div><div>I'm sorry, but I am not able to browse the internet to answer your question as I am a large language model trained by OpenAI. My knowledge is based solely on the text that I have been trained on, which has a cutoff date of 2021. I do not have any additional information beyond what I have been trained on.</div><div><br /></div><div>6. Compared to most other wealthy countries, the US has which of the following?</div><div>a. More hospital days</div><div>*b. More money spent</div><div>c. More nurses</div><div>d. More physicians</div><div><br /></div><div>Compared to most other wealthy countries, the US has more money spent on healthcare. The US spends more on healthcare as a percentage of its gross domestic product (GDP) than any other country in the world. According to data from the World Health Organization, the US spent 17.2% of its GDP on healthcare in 2018, compared to an average of 9.6% for other high-income countries. This high level of spending is due in part to the relatively high cost of healthcare services in the US, as well as the large number of uninsured and underinsured individuals in the country.</div><div><br /></div><div>7. Which of the following is not part of the original Affordable Care Act (ACA)?</div><div>*a. Incentives for meaningful use of electronic health records</div><div>b. Mandate that individuals buy health insurance</div><div>c. Outlawing of insurance denial due to pre-existing medical conditions</div><div>d. Programs to establish Accountable Care Organizations (ACOs)</div><div><br /></div><div>Programs to establish Accountable Care Organizations (ACOs) are not part of the original Affordable Care Act (ACA). ACOs are groups of healthcare providers who come together to coordinate care for a specific group of patients, with the goal of improving the quality of care and reducing costs. ACOs were introduced as part of the Medicare program under the Patient Protection and Affordable Care Act of 2010, but were not part of the original ACA legislation. The ACA included a number of other provisions, such as incentives for the meaningful use of electronic health records, a mandate that individuals buy health insurance, and the outlawing of insurance denial due to pre-existing medical conditions.</div><div><br /></div><div>8. About 50% of US healthcare spending is accounted for by which proportion of the population?</div><div>a. 1%</div><div>*b. 5%</div><div>c. 25%</div><div>d. 50%</div><div><br /></div><div>Approximately 50% of US healthcare spending is accounted for by 5% of the population. This phenomenon, known as the "5/50 rule," refers to the fact that a small proportion of the population accounts for a disproportionate amount of healthcare spending. The 5/50 rule is often used to illustrate the unequal distribution of healthcare costs in the US, with a small group of individuals with chronic conditions or complex healthcare needs accounting for a significant portion of total spending. This pattern is driven in part by the high cost of treating certain conditions, such as cancer and heart disease, as well as the high cost of specialty drugs and other medical technologies.</div><div><br /></div><div>9. Which of the following applications does not adhere to Friedman’s fundamental theorem of informatics?</div><div>*a. A machine learning algorithm that processes radiology images and determines the diagnoses more accurately than a physician</div><div>b. A telemedicine system that allows remote interaction via videoconference</div><div>c. A reminder system to administer cancer screening exams on an appropriate schedule</div><div>d. An information retrieval (search) system that helps the user select terms to improve search results</div><div><br /></div><div>A telemedicine system that allows remote interaction via videoconference does not adhere to Friedman's fundamental theorem of informatics. The fundamental theorem of informatics, proposed by Edward H. Friedman, states that "the value of medical information is inversely proportional to the time and place of its creation and directly proportional to the time and place of its use." This theorem suggests that the value of medical information decreases as it becomes older and less relevant, and increases when it is used at the time and place where it is needed. A telemedicine system that allows remote interaction via videoconference does not necessarily adhere to this theorem, as it allows healthcare providers to access and use medical information from different times and places. An example of an application that adheres to the theorem would be a reminder system that administers cancer screening exams on an appropriate schedule, as it uses medical information at the time and place where it is needed to improve patient care.</div><div><br /></div><div>10. A patient undergoing an appendectomy at a hospital near his suburban home is most likely receiving what type of healthcare?</div><div>a. Primary care</div><div>*b. Secondary care</div><div>c. Tertiary care</div><div>d. Quaternary care</div><div><br /></div><div>A patient undergoing an appendectomy at a hospital near his suburban home is most likely receiving secondary care. Secondary care refers to healthcare services that are provided by specialized medical professionals, such as surgeons, specialists, and other healthcare providers. These services are typically provided in hospitals or other specialized medical facilities, and often require referral from a primary care provider. An appendectomy, which is a surgical procedure to remove the appendix, is an example of a medical procedure that is typically performed in a hospital or other specialized medical facility, and would be classified as secondary care. Primary care, on the other hand, refers to the first level of contact between patients and the healthcare system, and typically involves services such as preventive care, routine check-ups, and treatment of minor illnesses. Tertiary care and quaternary care are higher levels of specialized care that are provided for patients with complex or chronic conditions.</div></div><div><br /></div>William Hersh, MDhttp://www.blogger.com/profile/08252786698451811645noreply@blogger.com0tag:blogger.com,1999:blog-1801786186944630495.post-75286497255465063782022-12-11T04:12:00.000-08:002022-12-11T04:12:30.956-08:00 Initial Excursions with ChatGPT<p>Like many people, I found myself caught up this past week testing out <a href="https://chat.openai.com/chat">ChatGPT</a>, the new automated chat agent released by <a href="https://openai.com/">OpenAI</a>. It seems like many people across the Internet are using it to test out its functioning in the kind of work they do, whether they are <a href="https://www.nytimes.com/2022/12/05/technology/chatgpt-ai-twitter.html">writers</a>, <a href="https://www.theverge.com/2022/12/5/23493932/chatgpt-ai-generated-answers-temporarily-banned-stack-overflow-llms-dangers">programmers</a>, or my fellow informatics researchers.</p><p>Naturally, I tested ChatGPT doing what I enjoy most, namely teaching. My first test was to enter the learning objectives for my well-known <a href="https://dmice.ohsu.edu/hersh/10x10.html">introductory biomedical and health informatics course</a>, aka, 10x10 ("ten by ten"). Being the academic that I am, I was also curious what ChatGPT might report for my contributions to the informatics field. The transcript of my interaction is provided below, but first some commentary.</p><p>ChatGPT did a much better job with my learning objectives than its report of my accomplishments in informatics. In fact, it got quite a bit wrong about my own work. For example, it reported me as Editor of <a href="https://academic.oup.com/jamia/">Journal of the American Medical Informatics Association (JAMIA)</a> and <a href="https://www.jmir.org/">Journal of Medical Internet Research ( JMIR)</a>, something my colleagues Sue Bakken and Gunther Eysenbach might dispute. It also gave erroneous information about books that I "authored." While I have edited and authored my textbook <a href="http://www.informaticsbook.info/">Health Informatics: Practical Guide</a>, authored my <a href="https://link.springer.com/book/10.1007/978-3-030-47686-1">textbook on information retrieval</a>, and contributed chapters to many other books, I have no involvement whatsoever in the books listed by ChatGPT, <a href="https://canadianscholars.ca/book/introduction-to-health-informatics/">Introduction to Health Informatics</a> or <a href="https://www.jblearning.com/catalog/productdetails/9781284036114">Essentials of Health Information Systems and Technology</a>. ChatGPT also listed first among my accomplishments in the field, "work on the development and implementation of EHRs." While I teach a great deal about the electronic health records (EHRs), it would be a stretch to say I am involved in their development and implementation. Clearly ChapGPT is an inferior alternative to Googling oneself.</p><p>For the learning objectives, ChatGPT did a little better, and gave reasonable, if incomplete, discussions on the topics in each. It still got some facts wrong, and also omitted what I might consider important details one might want to see in a succinct summary. Perhaps more importantly, ChatGPT provided no citations for what it said, which could allow a learner to check the interpretation or pursue more detail about the topic. I suppose the good news is that one still would do better to get more exhaustive text, figures, and citations covered in the 10x10 course (samples of which can be viewed at my site, <a href="http://informatics.health">informatics.health</a>).</p><p>These are obviously early days for tools such as ChatGPT, and no doubt they will improve over time. But just as machine learning algorithms are not quite ready to replace human clinicians, they also are not quite ready to overtake other human tasks, such as teaching, at least not yet.</p><p>ChatGPT Transcript - December 10, 2022 (my text entry in italics)</p><p><i>Define biomedical and health informatics, the terms related to it, and its role in health, healthcare, public health, and biomedical research</i></p><p>Biomedical and health informatics is a field that focuses on the use of technology and information to improve the delivery of healthcare, public health, and biomedical research. It is an interdisciplinary field that combines computer science, data science, and healthcare to develop and use information systems and technology to manage, analyze, and interpret health-related data.</p><p>Some of the terms related to this field include electronic health records (EHRs), health information technology (HIT), health information management (HIM), and health informatics. These terms all refer to different aspects of using information and technology in the healthcare and public health sectors.</p><p>The role of biomedical and health informatics in healthcare is to improve the efficiency and effectiveness of care delivery by using technology and information systems to manage and analyze health-related data. This can include the use of EHRs to manage patient records, the use of HIT to monitor and manage the health of patients remotely, and the use of HIM to ensure the quality and security of health information.</p><p>In public health, biomedical and health informatics can be used to monitor and analyze population-level health data to identify trends and patterns, and to develop interventions and policies to improve the health of the population.</p><p>In biomedical research, this field can be used to manage and analyze large amounts of data from clinical trials, genetic studies, and other research to advance our understanding of health and disease.</p><p>Overall, the goal of biomedical and health informatics is to use technology and information to improve the quality, efficiency, and accessibility of healthcare, public health, and biomedical research.</p><p><i>Describe the major historical events of biomedical and health informatics</i></p><p>The field of biomedical and health informatics has a long and rich history, with many important events that have shaped its development. Some of the major historical events in this field include:<br /></p><ol style="text-align: left;"><li>The development of electronic medical records (EMRs) in the 1950s, which marked the beginning of the use of computers in healthcare to store and manage patient information.</li><li>The establishment of the National Library of Medicine (NLM) in the United States in 1956, which became a major repository for health-related information and a driving force in the development of health informatics.</li><li>The creation of the first Health Information Management (HIM) program at the University of Minnesota in 1967, which marked the beginning of formal education and training in health informatics.</li><li>The development of the first clinical decision support system (CDSS) in the 1970s, which used computers to assist doctors in making diagnostic and treatment decisions.</li><li>The introduction of the first personal health record (PHR) in the 1980s, which allowed patients to manage their own health information and access it from any location.</li><li>The widespread adoption of the internet in the 1990s, which facilitated the sharing and dissemination of health-related information and the development of online health communities.</li><li>The passage of the Health Insurance Portability and Accountability Act (HIPAA) in 1996, which established national standards for the privacy and security of health information.</li><li>The growth of mobile health (mHealth) technologies in the 2000s, which enabled the use of smartphones and other devices to monitor and manage health-related information.</li></ol><p></p><p>Overall, these and other historical events have played a significant role in the development and evolution of biomedical and health informatics as a field, and have had a profound impact on the way healthcare is delivered and research is conducted.</p><p><i>Discuss the major problems in healthcare motivating use of biomedical and health informatics</i></p><p>There are many challenges and problems facing the healthcare industry today that are motivating the use of biomedical and health informatics. Some of the major problems in healthcare that are driving the use of this field include:</p><p></p><ol style="text-align: left;"><li>The increasing cost of healthcare: Healthcare costs are rising rapidly, and many people are struggling to afford the care they need. Biomedical and health informatics can help to reduce these costs by using technology and data analysis to improve the efficiency and effectiveness of care delivery.</li><li>The growing burden of chronic diseases: Chronic diseases, such as diabetes, heart disease, and cancer, are becoming more prevalent and are a major cause of morbidity and mortality. Biomedical and health informatics can be used to monitor and manage these conditions, and to identify and intervene early to prevent their progression.</li><li>The shortage of healthcare workers: There is a shortage of healthcare workers in many parts of the world, which is putting a strain on the healthcare system and making it difficult for people to access care. Biomedical and health informatics can help to alleviate this shortage by enabling healthcare workers to work more efficiently and by supporting remote care delivery.</li><li>The need for better population health management: Many public health challenges, such as obesity, substance abuse, and infectious diseases, require a population-level approach to prevention and management. Biomedical and health informatics can be used to monitor and analyze population-level health data, and to develop and implement interventions to improve the health of the population.</li><li>The need for better data management and analysis: The healthcare industry generates vast amounts of data, but much of it is unstructured and difficult to manage and analyze. Biomedical and health informatics can help to organize and analyze this data to extract valuable insights and inform decision-making.</li></ol><p></p><p>Overall, the major problems in healthcare are driving the use of biomedical and health informatics to improve the quality, efficiency, and accessibility of healthcare and to address some of the most pressing challenges facing the healthcare industry.</p><p><i>Compare and contrast the roles of various individuals in the health information technology workforce</i></p><p>The health information technology (HIT) workforce includes a wide range of individuals with different roles and responsibilities. Some of the major roles in the HIT workforce include:</p><p></p><ol style="text-align: left;"><li>Health information managers (HIMs): HIMs are responsible for the management and analysis of health-related data, including the design and implementation of health information systems and the development and maintenance of electronic health records (EHRs).</li><li>Health information technologists (HITs): HITs are responsible for the technical aspects of health information systems, including the design, development, and maintenance of software and hardware systems used in healthcare.</li><li>Clinical informaticists: Clinical informaticists are healthcare professionals with expertise in both medicine and health informatics. They use their knowledge of both fields to improve the quality and safety of care delivery, and to support the use of health information technology in clinical settings.</li><li>Health information exchange (HIE) coordinators: HIE coordinators are responsible for facilitating the exchange of health-related information between different healthcare organizations and systems. This includes ensuring the interoperability of different systems, and the security and privacy of health information.</li><li>Chief information officers (CIOs): CIOs are responsible for the overall management of information technology in healthcare organizations, including the development and implementation of information technology strategies and the supervision of IT staff.</li></ol><p></p><p>Overall, the roles in the HIT workforce are diverse and varied, and individuals in these roles work together to support the use of health information technology in healthcare and public health.</p><p><i>Find and access the major sources of information for biomedical and health informatics in the scientific literature and on the World Wide Web</i></p><p>There are many sources of information for biomedical and health informatics in the scientific literature and on the World Wide Web. Some of the major sources of information for this field include:</p><p></p><ol style="text-align: left;"><li>PubMed: PubMed is a free database of biomedical literature maintained by the National Institutes of Health (NIH) in the United States. It contains millions of abstracts and full-text articles from scientific journals, and is a valuable resource for researchers and practitioners in biomedical and health informatics.</li><li>IEEE Xplore Digital Library: The IEEE Xplore Digital Library is a comprehensive collection of technical literature in electrical engineering, computer science, and related fields. It contains many articles and conference proceedings on topics related to biomedical and health informatics, and is a valuable source of information for researchers in this field.</li><li>Health Information and Libraries Journal: The Health Information and Libraries Journal (HILJ) is a peer-reviewed journal published by the Health Libraries Group of the Chartered Institute of Library and Information Professionals (CILIP) in the United Kingdom. It covers a wide range of topics related to health information and health informatics, and is a valuable source of information for librarians and other professionals working in this field.</li><li>Journal of the American Medical Informatics Association (JAMIA): JAMIA is a peer-reviewed journal published by the American Medical Informatics Association (AMIA). It covers a wide range of topics related to health informatics, including the use of technology and information to improve the delivery of healthcare and public health.</li><li>Health IT Buzz: Health IT Buzz is a blog maintained by the Office of the National Coordinator for Health Information Technology (ONC) in the United States. It provides news and updates on health information technology and health informatics, and is a valuable source of information for those interested in this field.</li></ol><p></p><p>Overall, these and other sources of information are valuable for researchers and practitioners in biomedical and health informatics, and provide a wealth of information on the latest developments and trends in this field.</p><p><i>What are the contributions of William Hersh to informatics?</i></p><p>William Hersh is a well-known figure in the field of informatics. He is a professor and the chair of the Department of Medical Informatics & Clinical Epidemiology at the Oregon Health & Science University School of Medicine.</p><p>Some of Hersh's notable contributions to the field of informatics include his work on the development and implementation of electronic health records (EHRs), the use of natural language processing for clinical text analysis, and the integration of clinical research data with EHRs. He has also contributed to the development of informatics curricula and has written extensively on the subject.</p><p>Hersh is the author of several books on informatics, including "Introduction to Health Informatics" and "Essentials of Health Information Systems and Technology". He is also the editor-in-chief of the Journal of the American Medical Informatics Association and the Journal of Medical Internet Research.</p><p>Overall, Hersh has made many important contributions to the field of informatics, including his work on the development and implementation of EHRs, the use of natural language processing for clinical text analysis, and the integration of clinical research data with EHRs.</p><style class="WebKit-mso-list-quirks-style">
<!--
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-parent:"";
margin:0in;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Noto Serif",serif;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman \(Body CS\)";}
.MsoChpDefault
{mso-style-type:export-only;
mso-default-props:yes;
font-family:"Noto Serif",serif;
mso-ascii-font-family:"Noto Serif";
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:"Noto Serif";
mso-bidi-font-family:"Times New Roman \(Body CS\)";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;
mso-header-margin:.5in;
mso-footer-margin:.5in;
mso-paper-source:0;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:203566233;
mso-list-template-ids:1024068984;}
@list l1
{mso-list-id:738017694;
mso-list-template-ids:41430370;}
@list l2
{mso-list-id:1418284659;
mso-list-template-ids:-1388932858;}
@list l3
{mso-list-id:1573925828;
mso-list-template-ids:1569614684;}
-->
</style>William Hersh, MDhttp://www.blogger.com/profile/08252786698451811645noreply@blogger.com0tag:blogger.com,1999:blog-1801786186944630495.post-82519634308403499082022-10-12T12:10:00.000-07:002022-10-12T12:10:02.353-07:00The Most Profound Event in My Informatics Career? Unquestionably the HITECH Act<p>I recently took part in a small workshop exploring the benefits and challenges for artificial intelligence in medicine. Many of the participants were innovative medical educators, and most of them were still practicing clinical medicine. Although the workshop covered a broad range of topics, one theme stood out to me, perhaps because of my role as an academic informatician. In a nutshell, when we arrived at the topic of implementing AI systems in clinical practice, many laments were expressed about their impact on clinicians and insuring that the use of AI "cannot be as awful as the EHR."</p><p>As one who has been working in biomedical and health informatics for over three decades, this discussion about impact on practice and the experience of the electronic health record (EHR) drove home that unquestionably the most profound event in informatics during my entire career has been the the Health Information Technology for Economic and Clinical Health (HITECH) Act. Our field was changed substantially with the passage of the HITECH Act as part of the American Recovery and Reinvestment Act (ARRA) in 2009, leading our present world to be substantially different from the world before it. I also find that many who lament about the negative impacts of the EHR on healthcare are not aware of the history of HITECH.</p><p>Those of us who have been around long enough can remember informatics before HITECH. While EHRs had been adopted by many healthcare organizations, including my own, overall use was modest. We were also encouraged, especially in the decade of the 2000s before HITECH, by the growing amount of research published showing that the EHR and associated clinical decision support functionality led to error reduction, cost savings, and improved outcomes. Indeed, a systematic review in 2006 summarized the mostly positive studies, although noting they mostly came from "health IT exemplar" institutions [1]. An updated review in 2009 found similar results [2]. My own view at the time was captured from my <a href="https://www.npr.org/templates/story/story.php?storyId=4486530">participation in a panel</a> on the National Public Radio (NPR) show, Talk of the Nation - Science Friday, in 2005.</p><p>Another commentary published in 2010 noted that four "tribes" of healthcare improvement - focused on quality improvement, payment reform , consumer engagement, and health IT - were coalescing in the run-up to health care reform under President Barack Obama, for which HITECH was described by some as a "down payment" [3]. And the pre-HITECH enthusiasm all reached a boil when the ARRA legislation included the HITECH Act and its $30+ billion for incentives for EHR adoption [4].</p><p>Here is how I concluded <a href="https://informaticsprofessor.blogspot.com/2010/01/informatics-now-lives-in-hitech-world.html">a post in this blog from early 2010 highlighting the details of the HITECH Act</a>:</p><blockquote>This is a defining moment for the informatics field. Never before has such money and attention been lavished on it. HITECH provides a clear challenge for the field to 'get it right.' It will be interesting to look back on this time in the years ahead and see what worked and did not work. Whatever does happen, it is clear that informatics lives in a HITECH world now.</blockquote><p></p><p>To many in informatics, the HITECH Act may now be a distant event, and some in the field who are younger may not even had yet started working in it. But clearly the informatics world is much different now. Informatics is certainly more prominent in operational healthcare settings. Ever increasing numbers of healthcare organizations have anointed clinical leadership devoted to informatics [5].</p><p>In retrospect, the widespread EHR adoption of HITECH was carried out too hurriedly, and glossed over some major problems, not the least of which was inadequate adherence to data standards and interoperability. Indeed, it can be said that the health IT components of the 21st Century Cures Act, passed with a bipartisan majority in 2016, were some necessary corrections to the problems emanating from the HITECH Act [6]. The Cures Rule enshrining SMART on FHIR for interoperability and information blocking prohibitions to facilitate movement of data can be viewed from that perspective.</p><p>Whether deserved or not, the informatics community is included in the blame for the facts that EHRs slow clinicians down [7, 8], fill the record with questionable and duplicative information [9], and contribute to clinician burnout [10, 11]. The informatics field should accept part of the blame but must also be part of the solution. Actions such as the <a href="https://amia.org/about-amia/amia-25x5">AMIA-led 25x5 initiative</a> to reduce the burden of EHR documentation to 25% of its current level within five years are important. I also believe there are technologies that will improve the capture of data and clinical thought processes into the EHR, such as voice recognition and reworking of the user interface to facilitate care rather than just documenting it.</p><p>Clearly the HITECH Act was the most profound event of our field in modern times. The informatics world before and after HITECH is very different, for better or worse. Ever the informatics optimist, there is great opportunity to improve the use of EHR and other systems to deliver modern healthcare. I am confident we will rise to the challenge.</p><p>References</p><p>1. Chaudhry, B., Wang, J., Wu, S., Maglione, M., Mojica, W., Roth, E., Morton, S.C., Shekelle, P.G., 2006. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med 144, 742–752.</p><p>2. Goldzweig, C.L., Towfigh, A., Maglione, M., Shekelle, P.G., 2009. Costs and benefits of health information technology: new trends from the literature. Health Aff (Millwood) 28, w282-293.</p><p>3. McKethan, A., Brammer, C., 2010. Uniting the tribes of health system improvement. Am J Manag Care 16, SP13-18.</p><p>4. Blumenthal, D., 2011. Implementation of the federal health information technology initiative. N Engl J Med 365, 2426–2431.</p><p>5. Kannry, J., Sengstack, P., Thyvalikakath, T.P., Poikonen, J., Middleton, B., Payne, T., Lehmann, C.U., 2016. The Chief Clinical Informatics Officer (CCIO): AMIA Task Force Report on CCIO Knowledge, Education, and Skillset Requirements. Appl Clin Inform 7, 143–176.</p><p>6. Kesselheim, A.S., Avorn, J., 2017. New “21st Century Cures” Legislation: Speed and Ease vs Science. JAMA 317, 581–582.</p><p>7. Tutty, M.A., Carlasare, L.E., Lloyd, S., Sinsky, C.A., 2019. The complex case of EHRs: examining the factors impacting the EHR user experience. J Am Med Inform Assoc 26, 673–677.</p><p>8. Zheng, K., Ratwani, R.M., Adler-Milstein, J., 2020. Studying Workflow and Workarounds in Electronic Health Record-Supported Work to Improve Health System Performance. Ann Intern Med 172, S116–S122.</p><p>9. Steinkamp, J., Kantrowitz, J.J., Airan-Javia, S., 2022. Prevalence and Sources of Duplicate Information in the Electronic Medical Record. JAMA Netw Open 5, e2233348.</p><p>10. Gardner, R.L., Cooper, E., Haskell, J., Harris, D.A., Poplau, S., Kroth, P.J., Linzer, M., 2019. Physician stress and burnout: the impact of health information technology. J Am Med Inform Assoc 26, 106–114.</p><p>11. Murthy, V., 2022. Health Worker Burnout — Current Priorities of the U.S. Surgeon General [WWW Document]. <a href="https://www.hhs.gov/surgeongeneral/priorities/health-worker-burnout">https://www.hhs.gov/surgeongeneral/priorities/health-worker-burnout</a>. </p>William Hersh, MDhttp://www.blogger.com/profile/08252786698451811645noreply@blogger.com0tag:blogger.com,1999:blog-1801786186944630495.post-47464218655008814872022-09-18T21:23:00.002-07:002022-10-12T12:10:38.245-07:00A New Pathway For My Career in Biomedical and Health Informatics<p>This year, 2022, marks a quarter-century of my leadership of academic informatics at <a href="https://www.ohsu.edu/">Oregon Health & Science University (OHSU)</a>. I have served for 19 years as the one and only Chair of the <a href="https://www.ohsu.edu/school-of-medicine/medical-informatics-and-clinical-epidemiology">Department of Medical Informatics & Clinical Epidemiology (DMICE)</a> in the <a href="https://www.ohsu.edu/school-of-medicine">OHSU School of Medicine</a>. I served six years before that as Head of its predecessor free-standing Division of Medical Informatics & Outcomes Research, which was established in 1997.</p><p>I now plan to step down as Chair of DMICE while continuing at OHSU as a Professor in DMICE, focusing my activities on research, education, writing, and mentoring. I will support the new Chair and prioritize the continued success of the department.</p><p>I am pleased to hand off the department at a time when it is doing well academically and financially. The faculty are productive and well-funded in their research, and our educational programs continue to attract strong enrollment and prepare students for diverse careers in our field.</p><p>My own research and educational activities are also productive and well-funded. I plan to continue my research in applying information retrieval (IR, also known as search) methods to tasks such as patient cohort discovery though data from the electronic health record (EHR) for which I am funded with an <a href="https://reporter.nih.gov/search/sMMI6GJPaUWxrbqsVWcCBA/project-details/10450805">R01 grant from the National Library of Medicine</a>. I also intend to continue working in the evaluation of machine learning for clinical applications, mainly focused on diagnosis of rare diseases through EHR data.</p><p>I will also continue teaching both in the <a href="https://www.ohsu.edu/school-of-medicine/medical-informatics-and-clinical-epidemiology/education">OHSU Biomedical Informatics Graduate Program</a> as well as in external collaborations, such as with the <a href="https://amia.org/education-events/10x10-virtual-courses/10x10-oregon-health-science-university">American Medical Informatics Association (AMIA)</a> in the <a href="https://dmice.ohsu.edu/hersh/10x10.html">10x10 ("ten by ten") program</a> and the <a href="https://amia.org/education-events/clinical-informatics-board-review-course">Clinical Informatics Board Review Course (CIBRC)</a>. I will also continue existing and new National Institutes of Health (NIH)-funded educational activities, including our recently renewed <a href="https://www.nlm.nih.gov/ep/GrantTrainInstitute.html">NLM T15 Training Grant</a>, a new <a href="https://reporter.nih.gov/search/d-AztXxEsUqNXV6Dl37K7w/project-details/10630618">NLM R25 establishing a new college summer internship for students historically underrepresented in our field</a>, the <a href="https://dsi-africa.org/">Data Science Initiative for Africa</a> in <a href="https://reporter.nih.gov/search/sMMI6GJPaUWxrbqsVWcCBA/project-details/10490403">collaboration with the University of Cape Town</a>, and the new <a href="https://news.ohsu.edu/2022/09/13/ohsu-experts-tapped-for-national-artificial-intelligence-initiative">Bridge2AI Program</a>.</p>William Hersh, MDhttp://www.blogger.com/profile/08252786698451811645noreply@blogger.com0tag:blogger.com,1999:blog-1801786186944630495.post-70248161983143776762022-08-13T00:21:00.001-07:002022-08-13T00:32:06.319-07:00Arrival of 8th Edition of Health Informatics: Practical Guide<p>I am pleased to announce publication of the 8th Edition of <i>Health Informatics: Practical Guide</i>. The goal of this textbook is to provide a comprehensive introduction to the field of health informatics (also known as biomedical and health informatics or just plain informatics) for students. The book can also serve as a current update for those who are already practitioners. As in any professional field, there is a knowledge base and history of scientific progress in informatics that informs current practice, and this book aims to impart that.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhW6aBpIyfnvrEkzDBuDP1cI85-q_kfOI903Q6uELxd4x49fnm-QBI7IEfm21VJjGa0kHXNsk8G84_bxSPTaz_xhJapFasDSptEx-3rmBGqtnviRCcB5sbo4vBsC_xS_IWmytH7HBykYnHeX5LmibhEjwkbhJ9xM0zHx47daV9E5D4EcTinKHkPIEnjOA/s3250/book.jpeg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="3250" data-original-width="2849" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhW6aBpIyfnvrEkzDBuDP1cI85-q_kfOI903Q6uELxd4x49fnm-QBI7IEfm21VJjGa0kHXNsk8G84_bxSPTaz_xhJapFasDSptEx-3rmBGqtnviRCcB5sbo4vBsC_xS_IWmytH7HBykYnHeX5LmibhEjwkbhJ9xM0zHx47daV9E5D4EcTinKHkPIEnjOA/w281-h320/book.jpeg" width="281" /></a></div><p>The impetus for this book came from <a href="https://www.linkedin.com/in/robert-hoyt-md-facp-famia-abpm-ci-7ab96413/">Robert Hoyt, MD</a>, who served as its lead Editor for the first seven editions. In preparing the seventh edition, Dr. Hoyt engaged me as a Co-Editor, with the plan for me to become the Editor of the 8th edition. Dr. Hoyt and his wife <a href="https://health.usnews.com/doctors/ann-yoshihashi-802955">Ann Yoshihashi, MD</a> have been great mentors in the process of putting a book like this together. As with the previous editions, the book is available in print and eBook formats.</p><p>This book is aimed at those who wish to learn about the field of biomedical and health informatics. The content is almost completely redone from previous editions and roughly follows a course I have taught for three decades to about 5000 students in a graduate course at Oregon Health & Science University, the American Medical Informatics Association 10x10 ("ten by ten") program, and a number of other derivative courses for medical students, graduate students, and health IT professionals. I have engaged a variety of other authors for many of the chapters to give additional expertise and perspective. This edition of the book is completely rewritten from earlier editions, reflecting the profound change and progress in the field.</p><p>I have established a <a href="http://www.informaticsbook.info/">Web site for the book</a>, which has links to purchasing the book in <a href="https://www.lulu.com/shop/william-hersh/health-informatics-practical-guide-8th-edition/paperback/product-ekrqez.html">print</a> and <a href="https://www.lulu.com/shop/william-hersh/health-informatics/ebook/product-qm48yq.html">eBook</a> formats from the publisher, Lulu.com. The book is also available from Amazon.com in <a href="https://www.amazon.com/Health-Informatics-Practical-Guide-8th/dp/1387854755/">print</a> and Kindle formats. The Web site also contains sample chapters in PDF and EPUB formats, and will maintain an Errata page of errors discovered in the book.</p><p>The chapters of the book include:</p><p></p><ol style="text-align: left;"><li>Introduction to Biomedical and Health Informatics</li><li>A Short History of Biomedical and Health Informatics</li><li>Computing Concepts for Biomedicine and Health</li><li>Electronic Health Records</li><li>Standards and Interoperability</li><li>Data Science and Artificial Intelligence</li><li>Clinical Decision Support</li><li>Natural Language Processing</li><li>The Role of Health Informatics in Safety, Quality, and Value</li><li>Health Information Exchange (HIE)</li><li>EHR System Selection and Implementation</li><li>Telemedicine and Telehealth</li><li>Privacy and Security</li><li>Information Retrieval (Search)</li><li>Clinical Research Informatics</li><li>Translational Bioinformatics</li><li>Nursing Informatics</li><li>Consumer Health Informatics</li><li>Public Health Informatics</li><li>Evidence-Based Medicine</li><li>Imaging Informatics</li><li>Ethical Issues in Health Informatics</li><li>Human-Computer Interaction</li></ol><p></p>William Hersh, MDhttp://www.blogger.com/profile/08252786698451811645noreply@blogger.com0tag:blogger.com,1999:blog-1801786186944630495.post-19884944249554915622022-07-01T05:42:00.003-07:002022-07-02T05:41:27.136-07:00Reflections on Publishing a Negative Study<div>Although scientists are supposed to be disinterested observers of the results of their work, the reality is that none of us really want to see a negative result from our research. This is especially the case in fields like biomedical and health informatics, where we aim to develop tools that can be used to help patients and/or clinicians with their health or for delivery of care. Nonetheless, it is critically important to publish such outcomes, especially if the research was methodologically sound. This is especially the case in currently hyped technologies such as machine learning.</div><div><br /></div><div>One of the challenges for negative studies is the myriad of potential reasons why they failed, which sometimes makes the true reason hard to pinpoint. Was it poor methods, inadequate data, or mis-application of the technology? Or more than one of those? But especially in the case of studies that involve human subjects who may be exposing themselves to risk, we must always remember the words of noted physician-scientist Iain Chalmers, who stated that failure to publish studies with patients is a form of scientific misconduct.(1)</div><div><br /></div><div>A couple years ago, my colleague Aaron Cohen and I received some funding from a pharmaceutical company that had developed a highly effective new treatment for a rare disease, acute intermittent porphyria (AIP).(2) The new treatment, although expensive, significantly reduces the severe and sometimes disabling manifestations of AIP. As such, tools to identify patients who might have the disease could significantly improve their lives. Our approach used data from their electronic health record (EHR).</div><div><br /></div><div>We were thrilled when our machine learning model was found to identify a group of patients with the classic presentation of AIP, yet the diagnosis had never been considered.(3) AIP is known to be one of those rare diseases that often goes undiagnosed for long periods of time. Diagnosis of these types of rare diseases is considered to be a major use case for machine learning in healthcare. From a collection of over 200,000 patient records in the research data warehouse at our institution, we manually reviewed the top 100 ranking patients identified by the model. Twenty-two of these patients were determined to have the manifestations of AIP but with no occurrence of the string "porph" anywhere in their record, i.e., not in lab tests, notes, or diagnoses. As the test to diagnose AHP is a relatively simple and inexpensive urine porphobilinogen test, we developed a clinical protocol to invite such patients for testing.</div><div><br /></div><div>We encountered a number of challenges in developing and implementing the clinical protocol. For example, what is the role of the patient's primary care or other provider in the decision to offer testing? Our institutional review board (IRB) determined that the process should a decision by the provider, who would then allow us to contact the patient to offer testing. Once given approval by the provider, we then had to convince the patients, many of them skeptical of the healthcare system long unable to diagnose their symptoms, to come in for testing. Another challenge in implementing the protocol was that it took place during the COVID-19 pandemic, when many people wanted to avoid healthcare settings, though by the time we were recruiting patients, the COVID-19 vaccines had started to become available.</div><div><br /></div><div>As a result, we could only convince 7 of the 22 patients to undergo the simple and free test we were offering them. And as noted in the paper, none of them were positive.(4) Because these patients did submit to a clinical protocol, and because we believe that any attempt to clinically validate machine learning is important, we decided to submit our results to a journal as a Brief Communication, i.e., not the definitive study but clearly worth reporting. The report has now been published.</div><div><br /></div><div>Clearly there are many possible reasons for our failure to diagnose any new cases of AIP. Not the least of these is the fact that AIP is a rare disease, and there may have been few if any new cases to be found. Certainly if we had more resources or time, we could have invited for testing more people beyond the first 100 cases we evaluated who may also have been found to have the classic presentation yet for whom the diagnosis was never considered. It is also possible that our machine learning model could have been better at identifying true cases of the disease. There may have been confounders in that those actually coming for testing were not the most likely to have the diagnosis.</div><div><br /></div><div>Could this study have benefited from proactive genotyping? There may be a role for gene sequencing in this situation, but AIP is a condition of incomplete penetrance, i.e., just because one has the genotype does not mean they will develop the manifestations of the disease. In other words, genotyping would not be enough.</div><div><br /></div><div>My main hope is that this contribution shows that we should be pursuing such studies aiming to apply machine learning in real-world clinical settings. As one who has been <a href="https://informaticsprofessor.blogspot.com/2021/07/translational-artificial-intelligence.html">critical that machine learning has not been "translational" enough</a>, I hope there will be a role for larger and more comprehensive studies of many different uses that have been found to be beneficial in model-building studies.</div><div><br /></div><div>References</div><div><br /></div><div><div>1. Chalmers, I., 1990. Underreporting research is scientific misconduct. <i>JAMA</i> 263, 1405–1408.</div><div><br /></div><div>2. Balwani, M., Sardh, E., Ventura, P., Peiró, P.A., Rees, D.C., Stölzel, U., Bissell, D.M., Bonkovsky, H.L., Windyga, J., Anderson, K.E., Parker, C., Silver, S.M., Keel, S.B., Wang, J.-D., Stein, P.E., Harper, P., Vassiliou, D., Wang, B., Phillips, J., Ivanova, A., Langendonk, J.G., Kauppinen, R., Minder, E., Horie, Y., Penz, C., Chen, J., Liu, S., Ko, J.J., Sweetser, M.T., Garg, P., Vaishnaw, A., Kim, J.B., Simon, A.R., Gouya, L., ENVISION Investigators, 2020. <a href="https://doi.org/10.1056/NEJMoa1913147">Phase 3 Trial of RNAi Therapeutic Givosiran for Acute Intermittent Porphyria</a>. <i>N Engl J Med</i> 382, 2289–2301.</div><div><br /></div><div>3. Cohen, A.M., Chamberlin, S., Deloughery, T., Nguyen, M., Bedrick, S., Meninger, S., Ko, J.J., Amin, J.J., Wei, A.J., Hersh, W., 2020. <a href="https://doi.org/10.1371/journal.pone.0235574">Detecting rare diseases in electronic health records using machine learning and knowledge engineering: Case study of acute hepatic porphyria</a>. <i>PLoS ONE</i> 15, e0235574.</div><div><br /></div><div>4. Hersh, W.R., Cohen, A.M., Nguyen, M.M., Bensching, K.L., Deloughery, T.G., 2022. <a href="https://doi.org/10.1093/jamiaopen/ooac053">Clinical study applying machine learning to detect a rare disease: results and lessons learned</a>. <i>JAMIA Open</i> 5, ooac053.</div><div><br /></div></div>William Hersh, MDhttp://www.blogger.com/profile/08252786698451811645noreply@blogger.com0tag:blogger.com,1999:blog-1801786186944630495.post-50244523711883608772022-06-29T06:20:00.004-07:002022-06-29T06:21:29.156-07:002022 Update of Informatics.Health<p>I am pleased to announce an update of my Web site that provides an introductory overview of biomedical and health informatics. Entitled, <i>What is Biomedical & Health Informatics?</i>, I created this site two decades ago to provide an answer to that question I used to be asked (less so in modern times). I still maintain and keep the site up to date both to continue to provide an overview of the field as well as demonstrate a portfolio of some of the learning technology we use in our virtual courses in the <a href="https://www.ohsu.edu/school-of-medicine/medical-informatics-and-clinical-epidemiology/prospective-students">Biomedical Informatics Graduate Program at Oregon Health & Science University (OHSU)</a>.<br /><br />In recent years I was able to secure the domain name, <a href="http://Informatics.Health">Informatics.Health</a>.<br /><br />With the 2022 updating of my larger course that is offered in the <a href="https://dmice.ohsu.edu/hersh/10x10.html">American Medical Informatics Association (AMIA) 10x10 ("ten by ten") program</a>, I have now updated the content of the WhatIs site. The main part of the site is the 11 lecture segments on the following topics:<br /></p><ul style="text-align: left;"><li>What is Biomedical and Health Informatics? (1) (24:32)</li><li>What is Biomedical and Health Informatics? (2) (18:49)</li><li>A Short History of Biomedical and Health Informatics (22:30)</li><li>Resources for Field: Organizations, Information, Education (25:29) </li><li>Clinical Data (15:08)</li><li>Examples of the Electronic Health Record (EHR) (24:56)</li><li>Data Science and Artificial Intelligence (1) (14:15)</li><li>Data Science and Artificial Intelligence (2) (22:07)</li><li>Information Retrieval (Search) (23:18)</li><li>Information Retrieval Content (29:09)</li></ul><p>The lectures can be viewed on just about any Web platform, and work fine on mobile devices. The site also contains links to books, articles, organizations, and educational Web sites. The materials on the site are freely available and have been used by many educators and others.</p>William Hersh, MDhttp://www.blogger.com/profile/08252786698451811645noreply@blogger.com0tag:blogger.com,1999:blog-1801786186944630495.post-83722706306011045722022-04-06T05:03:00.006-07:002022-04-08T04:22:47.354-07:003000 by 2022 - A New Milestone for the 10x10 Course<p>One of the most enjoyable and impactful activities of my career has been the <a href="https://dmice.ohsu.edu/hersh/10x10.html">10x10 ("ten by ten") course</a>, which is an online introductory course in biomedical and health informatics that I teach in <a href="https://amia.org/education-events/10x10-virtual-courses/10x10-oregon-health-science-university">partnership with the American Medical Informatics Association (AMIA)</a>. The course recently reached a new milestone, surpassing 3000 people completing the course since its inception in 2005. <a href="https://dmice.ohsu.edu/hersh/10x10.html">My course from Oregon Health & Science University (OHSU)</a> was the original and is the largest course of the <a href="https://amia.org/education-events/amia-10x10-virtual-courses">AMIA 10x10 Program</a>.</p><p>The OHSU 10x10 course provides an up-to-date detailed introductory overview of biomedical and health informatics. The content delivery is online, followed by an optional in-person session, typically at an AMIA meeting, at its conclusion. About 10-15% of those completing the course take the optional final exam and pursue further study in the field, usually but not exclusively at OHSU.</p><div>The 10x10 program gets its name from the program's original goal in 2005 to train 10,000 people in biomedical and health informatics by the year 2010. The goal for 10,000 people came from a stated need by then-President of AMIA Dr. Charles Safran that there should be at least one physician and one nurse trained in informatics in each of the approximately 6000 hospitals in the US. I adapted an already-existing online introductory informatics course and we set the goal of 10,000 people completing the course by 2010, leading me to devise the moniker, 10x10. Even though 10,000 people did not come forward to take the course by 2010, <a href="https://informaticsprofessor.blogspot.com/2010/12/10x10-ten-by-ten-at-end-of-2010-and.html">about 1000 people did complete the OHSU offering by 2010</a>.</div><div> </div><div data-en-clipboard="true" data-pm-slice="1 1 []">After 2010, the 10x10 program <a href="https://informaticsprofessor.blogspot.com/2020/11/15-years-of-10x10-ten-by-ten.html">continued to attract enrollment</a>. The <a href="https://amia.org/education-events/amia-10x10-virtual-courses/amia-10x10-testimonials">positive feedback of those completing the course</a> has always been gratifying. Now, a new milestone has been reached with the cohort who recently completed the course, surpassing a total of 3000 individuals having completed the OHSU 10x10 course since 2005. A total of 111 offerings of the OHSU course have been provided since that time. While most of the course enrollment has come from the US, several international organizations have partnered to offer the course based in their countries, including Gateway Consulting (Singapore), King Saud University (Saudi Arabia), the Ministry of Health (Israel), and Abu Dhabi Health Services (SEHA, United Arab Emirates). There have also been partnerships to offer the course with domestic organizations in the US, including the American College of Emergency Physicians, the Academy of Nutrition and Dietetics (AND), the New York State Academy of Family Practice, the Mayo Clinic, the Centers for Disease Control and Prevention, and others. The table below shows the course partners, the number of offerings with them, and the number of people completing the course.</div><div data-en-clipboard="true" data-pm-slice="1 1 []"> </div><div data-en-clipboard="true" data-pm-slice="1 1 []"><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiolV01VXfAnig5njcXYCtK5GvtwlBOx2BwrGHUjUE9Xz8nri6qnDy6R7zyOvY1GEEBHSCxXgkEn5N46wH2RBNs6kSeT3Drrj-WeEXY01rU4zVPgl0_Zi57hmQKYBy2-ZyBQcC7svWuSQBco_u-FUQbH_Baa5yupJ2vX2VdZ2UM46cor0qWuACA9HfLog/s1354/table.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1340" data-original-width="1354" height="317" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiolV01VXfAnig5njcXYCtK5GvtwlBOx2BwrGHUjUE9Xz8nri6qnDy6R7zyOvY1GEEBHSCxXgkEn5N46wH2RBNs6kSeT3Drrj-WeEXY01rU4zVPgl0_Zi57hmQKYBy2-ZyBQcC7svWuSQBco_u-FUQbH_Baa5yupJ2vX2VdZ2UM46cor0qWuACA9HfLog/w320-h317/table.png" width="320" /></a></div> </div><div>The figure below shows the enrollment by year. The largest amount of enrollment came during the heyday of the "meaningful use" era in the early 2010s, dipped somewhat later in the decade, but has continued strong since then.<div><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEil8O9ws_PZYjc8-bfGEBZzlJzSMJ-qgo-HSK9BZn7tIFegy7pOne8potZcrrjYPYO1UwAn-8gU9TQ743vxoy4Gt1TooZDj-3dqXCzJv2JnQsTbOVXekt4a2mQRODPM-OahFo406391igDtVZiZ_ueCSs7VsZpji9QB92jpMwWNQAl_L0VXiLlMylGFDA/s1429/figure.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1035" data-original-width="1429" height="232" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEil8O9ws_PZYjc8-bfGEBZzlJzSMJ-qgo-HSK9BZn7tIFegy7pOne8potZcrrjYPYO1UwAn-8gU9TQ743vxoy4Gt1TooZDj-3dqXCzJv2JnQsTbOVXekt4a2mQRODPM-OahFo406391igDtVZiZ_ueCSs7VsZpji9QB92jpMwWNQAl_L0VXiLlMylGFDA/w320-h232/figure.png" width="320" /></a></div><div> </div><div>Due to the online delivery of the course, it had no trouble continuing during the COVID-19 pandemic, although the in-person sessions at the end of the course were transitioned to virtual delivery. With the receding of the pandemic, those sessions have returned now to an in-person format. Of course, the content of the course was affected by the pandemic and its impacts on the field. However, the more fundamental topics of the field, including electronic health records, standards and interoperability, machine learning and artificial intelligence, and others have still been covered in a comprehensive and up-to-date manner.</div><div><br /></div><div>The course will continue to be offered beyond this milestone, and in fact a new offering is beginning this month. In-person sessions for those completing the course will take place at upcoming AMIA meetings this year, including the <a href="https://amia.org/education-events/amia-2022-clinical-informatics-conference">Clinical Informatics Conference</a> (Houston, TX - May 23-26) and the <a href="https://amia.org/education-events/annual-symposium">Annual Symposium</a> (Washington, DC - November 5-9).</div><div><br /></div><div>As the OHSU agreement with AMIA is non-mutually-exclusive (i.e., AMIA can offer other 10x10 courses and OHSU can use the course materials in other courses), the content of the OHSU 10x10 course has been delivered to students under other course names. The other major use of the content is in the course, <a href="https://ohsu.campusconcourse.com/view_syllabus?course_id=29535&public_mode=1">BMI 510/610</a>, which is the introductory course in the <a href="https://www.ohsu.edu/school-of-medicine/medical-informatics-and-clinical-epidemiology/health-and-clinical-informatics-0">Health & Clinical Informatics Major</a> of the <a href="https://www.ohsu.edu/school-of-medicine/medical-informatics-and-clinical-epidemiology/education">OHSU Biomedical Informatics Graduate Program</a>. This course has been offered since the OHSU graduate program was launched in 1996 and every academic quarter since 2000. After the most recent academic term, 1598 people have completed this instance of the course.</div><div><br /></div><div data-en-clipboard="true" data-pm-slice="1 1 []">The course content was also converted into a <a href="https://informaticsprofessor.blogspot.com/2020/04/virtual-informatics-course-for-medical.html">virtual two-week block course for medical students at the onset of the pandemic</a>. The course was made available to medical students within and outside OHSU, with 44 OHSU medical students participating in three offerings and 178 medical students from outside OHSU participating in eight offerings during 2020. This block course continues to be offered to OHSU medical students.<br /></div><div data-en-clipboard="true" data-pm-slice="1 1 []"> </div><div data-en-clipboard="true" data-pm-slice="1 1 []">Other offerings of the course have been provided to health systems, including Kaiser Permanente Northwest, Providence Health and Systems, and Bangkok (Thailand) Hospital. The course has also been offered to <a href="https://www.h3abionet.org/">H3ABioNet</a>, the Pan African Bioinformatics Network for the Human Heredity and Health in Africa (H3Africa) consortium, of which I serve on the <a href="https://www.h3abionet.org/about/sab">Scientific Advisory Board</a>.</div><div data-en-clipboard="true" data-pm-slice="1 1 []"> </div><div data-en-clipboard="true" data-pm-slice="1 1 []">The course also inspired federal legislation in the US, leading to the <a href="http://www.govtrack.us/congress/bill.xpd?bill=h111-461">10,000 Trained by 2010 Act</a>, introduced by Rep. David Wu (D-OR). The bill was passed by the US House in the 111th Congress (2009-2010), and elements of it were incorporated into the HITECH Act. The HITECH program included funding for Health IT Workforce Development, including <a href="http://informaticsprofessor.blogspot.com/2010/04/ohsu-awarded-58-million-to-expand.html">grants awarded to OHSU</a>. <br /></div><div data-en-clipboard="true" data-pm-slice="1 1 []"><br /></div><div data-en-clipboard="true" data-pm-slice="1 1 []">Probably the most gratifying aspect of the course is the feedback I get from students who appreciate the broad but sufficiently deep content, i.e., getting the big picture while also learning the rationale behind it. I enjoy keeping in touch with those who have taken the course, especially when I run into them in airports, at conferences, or other places. I also value the motivation that the course provides to me in keeping up with advances across the entire field of biomedical and health informatics.<br /></div></div>William Hersh, MDhttp://www.blogger.com/profile/08252786698451811645noreply@blogger.com0