This week, our Department of Medical Informatics & Clinical Epidemiology (DMICE) at Oregon Health & Science University (OHSU) announced a new partnership with Epic Systems Corp., developer of the market-leading Epic electronic health record (EHR). The partnership will make the Epic EHR and associated tools available for research and educational purposes to OHSU informatics students and faculty. OHSU is the first academic informatics program to partner with Epic in this manner, in a process that Epic hopes can be exported to its other customers who also have academic informatics programs.
The partnership will entail setting up two laboratory environments at OHSU, one focused on research and the other on education. The research environment will provide access to the source-code level and enable investigation of areas such as usability, data analytics, simulation, interoperability, patient safety, and others. This platform will allow OHSU faculty and students to conduct research with the Epic EHR environment.
The educational environment will provide students in OHSU's biomedical informatics graduate program access to the Epic EHR and associated tools for learning purposes. Students in both OHSU's on-campus and distance-learning programs will be able to pursue coursework based on Epic's state-of-the-art EHR system. Educational activities will include learning to configure screens, implement clinical decision support, generate reports, and perform other front-end and back-end activities.
Every student of informatics who aspires to work professionally in the field should have experience with a state-of-the-art electronic health record system, including back-end functionality. This partnership provides the opportunity for our students, including those in our distance learning program, to obtain such experience. This will not only augment their learning but also make them more competitive for jobs when they graduate.
This project also represents one of the first milestones of OHSU's new Informatics Discovery Lab (IDL), which was described in this blog previously and is led by Aaron Cohen, MD, MS.
Once the laboratory is established, OHSU will provide support to other of Epic's academic customers in establishing a similar laboratory environment for their programs.
This blog maintains the thoughts on various topics related to biomedical and health informatics by Dr. William Hersh, Professor, Department of Medical Informatics & Clinical Epidemiology, Oregon Health & Science University.
Thursday, November 21, 2013
Monday, November 11, 2013
Continued Concerns for Building the Capacity of the Clinical Informatics Subspecialty
About a year ago, I wrote a posting expressing concerns about how we will build capacity for the clinical informatics subspecialty. A year later, the subspecialty has moved forward, with the first offering of the certification exam this past October as well as release of draft rules for fellowships accredited by the Accreditation Council for Graduate Medical Education (ACGME).
I still, however, have many of the concerns raised last year, especially now that some of the challenges for standing up ACGME-accredited fellowships have come into sharper focus with the release of the draft rules, and some of us actually trying to figure out how we will develop these fellowships within our own institutions. While the current "grandfathering" period is likely to result in a good starting number of certified individuals, the subspecialty will only succeed in the long run if there are enough training programs to meet the workforce needs of the subspecialty going forward. If only a small number of training programs develop, or they are very difficult for individuals to enter for reasons having nothing to do with their qualifications, then the subspecialty may not be sustianable in the long run.
I have had many of these concerns since the very idea of a subspecialty was first developed by the American Medical Informatics Association (AMIA) around 2006. While I still greatly support the notion of professional recognition of those who work in informatics, I nonetheless have concerns about trying to do so using the mechanism of the medical board subspecialty. In this posting, I will update my thinking on these concerns and also present an alternative for addressing them, even though I am well aware that it may be incompatible with the rules, or at least the traditions, of ACGME.
Background
For those not familiar with the conventional approach to medical subspecialty training, it is important to remember that clinical fellowships differ somewhat from the graduate education model under which most informatics education programs have historically operated. In graduate education, progress is made in units of courses. While most courses are based on specific subjects and/or competencies, there can also be courses for internship/practicum work, theses/dissertations, and integrative learning experiences. With the world's growing number of adult learners in all fields, graduate education can be pursued on a part-time basis, at the appropriate pace for the learner.
Medical training (including clinical fellowships), however, has historically progressed in time-based units, typically in units of years. As such, internal medicine or family medicine residencies are three years long, with other specialties having similar or longer training periods. Those who pursue subspecialty training do so in fellowships lasting one to several years. Sometimes a training program will be extended for research or other work, but almost always in one-year increments.
It is therefore no surprise that the draft ACGME rules propose a time-based approach to training. According to the draft rules, the clinical informatics fellowship will be two years long. While ACGME is currently reviewing public comments, including ours, it is unlikely they will change from that two-year length.
Challenges
Whereas my concerns about clinical informatics fellowships last year were conceptual, we now have real (draft) rules to figure out how to develop fellowships. My concerns are not lessened, and I see potential problems. These include the funding model of fellowships, the requirement to align in a given institution with a single medical specialty, and the arbitrary two-year time frame. I do believe, however, that there are means to accomplish the same ends by a different approach, which I will explain after elaborating more on my concerns.
The funding problem for clinical informatics fellowships will stem from the fact that these fellowships will be different from most other subspecialty fellowships. This derives from the regulations of the Centers for Medicare and Medical Services (CMS) that when one is a clinical fellow, he or she is viewed as a trainee and not allowed to perform any clinical practice without supervision, even if fully board-certificated in the area of practice.
In a typical clinical fellowship, this mostly makes sense, as the clinical work that a fellow is doing is related to the subspecialty in which he or she is training. A clinical informatics fellow will be functioning somewhat differently, as his or her fellowship work will be done in informatics, whereas he or she will be expected to maintain their clinical skills in the practice of the specialty in which he or she was originally board-certified. I am not aware of any fellowship that asks a clinical fellow to maintain practice in their original specialty. Cardiology or rheumatology fellows are not asked, for example, to maintain their practice of general internal medicine, which is the specialty in which they trained before entering their subspecialty fellowship.
Continuing this example, a person entering a cardiology or rheumatology fellowship after completing an internal medicine residency will perform his or her clinical work in their fellowship in cardiology or rheumatology. A clinical informatics fellow training after an internal medicine residency, on the other hand, will be doing his or her "clinical" work in informatics and practicing general internal medicine, mostly to maintain his or her clinical skills. A challenge is that this practice will need to be supervised as if the fellow were a trainee, even if he or she is already board-certified. This will incur a cost to institutions in that attending physicians are limited in the number of residents and fellows they can supervise at any one time. By requiring the clinical informatics fellow to have supervision, it is one less other resident or fellow that an attending physician can supervise under CMS rules.
This underscores the bigger challenge of figuring out how to fund the fellowship. Unlike a graduate education program, where someone (the student, a training grant, a scholarship, etc.) pays the tuition, which in turn supports the program, a clinical fellow earns a salary, which is funded by either CMS graduate medical education (GME) money or the institution itself. Most clinical departments in academic medical centers can fund such fellowships out of other revenues, including the practice revenues of the attending physicians. The challenge for clinical informatics fellows will be how to fund the fellows, not only their stipends, but also the other costs, such as faculty supervision time, tuition for the courses offered, scholarship expenses (e.g., travel to meetings), and administrative costs.
One option suggested by some is to allow the fellows to "moonlight," a tried and true way for medical trainees to augment their income. A problem with moonlighting for clinical experience and revenue is that it will disconnect the clinical informatics fellow clinically from the setting where he or she is doing their informatics training, thus making them less integrated with the environment whose informatics systems they are learning to improve.
A second major problem is the ACGME requirement for clinical informatics fellowships to be aligned with one of six (perhaps increasing to nine) residency programs. This is mainly being done for efficiency reasons, so that new residency review committees (RRCs) that review residencies and fellowships for accreditation will not need to be created. It is stressed that programs can accept anyone from any specialty, but in reality programs will likely resemble the specialties to whose programs they are administratively aligned, if for no other reason due to the normal give and take of academic medical center negotiations and politics.
A third major problem is the two-year time frame. While I do believe that most clinical informatics subspecialists are likely to require two years to complete their training, the arbitrary two-year time frame is at odds with the growing change in medical education from time-based tp competency-based education. Learning informatics is no more like "steeping tea" than medical education [1].
One of the outcomes from these problems is that it may be too difficult for there to be an adequate number of fellowship programs for qualified individuals from all medical specialties to make the subspecialty truly viable. If each fellowship has a one-off situation (i.e., only able or willing to take trainees from one or a small number of specialties) or must be funded, to use the words of my institution's chief medical officer, "creatively," this may undermine the long-term viability of the subspecialty.
Alternative Approach
I cannot criticize the proposed approach without offering an alternative, and I believe there are approaches that could be rigorous enough to ensure an equally if not more robust educational and training experience than the proposed fellowship model. It would no doubt test the boundaries of a tradition-bound organization like ACGME but could also show innovation reflective (and indeed required) of modern education.
We must remember that there will be three basic activities of clinical informatics subspecialty trainees:
Second, how would trainees get their practical hands-on project work? Again, many informatics programs, certainly ours, have developed mechanisms by which students can do internships or practicums in remote location through a combination of affiliation agreements, local mentoring, and remote supervision. While our program currently has students performing 3-6 months at a time of these, I see no reason why the practical experience could not be expanded to a year or longer. Strict guidelines for experience and both local and remote mentoring could be put in place to insure quality.
Finally, what about clinical practice? This may be easiest of all. Requiring a trainee to perform a certain volume of clinical practice, while adhering to all appropriate requirements for licensure and maintenance of certification, should be more than adequate to insure practice in their primary specialty. Many informatics distance learning students are already maintaining their clinical practices to maintain their livelihood. Making clinical practice explicit, instead of as something requiring supervision, will also allow training to be more financially viable for the fellow. Any costs of tuition and practical work could easily be offset by clinical practice revenue.
There would obviously need to be some sort of national infrastructure to set standards and monitor progress of clinical informatics trainees. There are any number of organizations that could perform this task, such as AMIA, and it could perhaps be a requirement of accreditation.
In fact, ACGME and the larger medical education community may learn from alternative approaches like this for training in other specialties. One major national concern these days is that number of residency positions for medical school graduates is not keeping up with the increases of medical school enrollment or, for that matter, the national need for physicians [3]. It is possible that alternative approaches like this could expand the capacity of all medical specialties and subspecialties, and not just clinical informatics.
References
1. Hodges BD, A tea-steeping or i-Doc model for medical education? Academic Medicine, 2010. 85: S34-S44.
2. Hersh WR, The full spectrum of biomedical informatics education at Oregon Health & Science University. Methods of Information in Medicine, 2007. 46: 80-83.
3. Iglehart JK, The residency mismatch. New England Journal of Medicine, 2013. 369: 297-299.
I still, however, have many of the concerns raised last year, especially now that some of the challenges for standing up ACGME-accredited fellowships have come into sharper focus with the release of the draft rules, and some of us actually trying to figure out how we will develop these fellowships within our own institutions. While the current "grandfathering" period is likely to result in a good starting number of certified individuals, the subspecialty will only succeed in the long run if there are enough training programs to meet the workforce needs of the subspecialty going forward. If only a small number of training programs develop, or they are very difficult for individuals to enter for reasons having nothing to do with their qualifications, then the subspecialty may not be sustianable in the long run.
I have had many of these concerns since the very idea of a subspecialty was first developed by the American Medical Informatics Association (AMIA) around 2006. While I still greatly support the notion of professional recognition of those who work in informatics, I nonetheless have concerns about trying to do so using the mechanism of the medical board subspecialty. In this posting, I will update my thinking on these concerns and also present an alternative for addressing them, even though I am well aware that it may be incompatible with the rules, or at least the traditions, of ACGME.
Background
For those not familiar with the conventional approach to medical subspecialty training, it is important to remember that clinical fellowships differ somewhat from the graduate education model under which most informatics education programs have historically operated. In graduate education, progress is made in units of courses. While most courses are based on specific subjects and/or competencies, there can also be courses for internship/practicum work, theses/dissertations, and integrative learning experiences. With the world's growing number of adult learners in all fields, graduate education can be pursued on a part-time basis, at the appropriate pace for the learner.
Medical training (including clinical fellowships), however, has historically progressed in time-based units, typically in units of years. As such, internal medicine or family medicine residencies are three years long, with other specialties having similar or longer training periods. Those who pursue subspecialty training do so in fellowships lasting one to several years. Sometimes a training program will be extended for research or other work, but almost always in one-year increments.
It is therefore no surprise that the draft ACGME rules propose a time-based approach to training. According to the draft rules, the clinical informatics fellowship will be two years long. While ACGME is currently reviewing public comments, including ours, it is unlikely they will change from that two-year length.
Challenges
Whereas my concerns about clinical informatics fellowships last year were conceptual, we now have real (draft) rules to figure out how to develop fellowships. My concerns are not lessened, and I see potential problems. These include the funding model of fellowships, the requirement to align in a given institution with a single medical specialty, and the arbitrary two-year time frame. I do believe, however, that there are means to accomplish the same ends by a different approach, which I will explain after elaborating more on my concerns.
The funding problem for clinical informatics fellowships will stem from the fact that these fellowships will be different from most other subspecialty fellowships. This derives from the regulations of the Centers for Medicare and Medical Services (CMS) that when one is a clinical fellow, he or she is viewed as a trainee and not allowed to perform any clinical practice without supervision, even if fully board-certificated in the area of practice.
In a typical clinical fellowship, this mostly makes sense, as the clinical work that a fellow is doing is related to the subspecialty in which he or she is training. A clinical informatics fellow will be functioning somewhat differently, as his or her fellowship work will be done in informatics, whereas he or she will be expected to maintain their clinical skills in the practice of the specialty in which he or she was originally board-certified. I am not aware of any fellowship that asks a clinical fellow to maintain practice in their original specialty. Cardiology or rheumatology fellows are not asked, for example, to maintain their practice of general internal medicine, which is the specialty in which they trained before entering their subspecialty fellowship.
Continuing this example, a person entering a cardiology or rheumatology fellowship after completing an internal medicine residency will perform his or her clinical work in their fellowship in cardiology or rheumatology. A clinical informatics fellow training after an internal medicine residency, on the other hand, will be doing his or her "clinical" work in informatics and practicing general internal medicine, mostly to maintain his or her clinical skills. A challenge is that this practice will need to be supervised as if the fellow were a trainee, even if he or she is already board-certified. This will incur a cost to institutions in that attending physicians are limited in the number of residents and fellows they can supervise at any one time. By requiring the clinical informatics fellow to have supervision, it is one less other resident or fellow that an attending physician can supervise under CMS rules.
This underscores the bigger challenge of figuring out how to fund the fellowship. Unlike a graduate education program, where someone (the student, a training grant, a scholarship, etc.) pays the tuition, which in turn supports the program, a clinical fellow earns a salary, which is funded by either CMS graduate medical education (GME) money or the institution itself. Most clinical departments in academic medical centers can fund such fellowships out of other revenues, including the practice revenues of the attending physicians. The challenge for clinical informatics fellows will be how to fund the fellows, not only their stipends, but also the other costs, such as faculty supervision time, tuition for the courses offered, scholarship expenses (e.g., travel to meetings), and administrative costs.
One option suggested by some is to allow the fellows to "moonlight," a tried and true way for medical trainees to augment their income. A problem with moonlighting for clinical experience and revenue is that it will disconnect the clinical informatics fellow clinically from the setting where he or she is doing their informatics training, thus making them less integrated with the environment whose informatics systems they are learning to improve.
A second major problem is the ACGME requirement for clinical informatics fellowships to be aligned with one of six (perhaps increasing to nine) residency programs. This is mainly being done for efficiency reasons, so that new residency review committees (RRCs) that review residencies and fellowships for accreditation will not need to be created. It is stressed that programs can accept anyone from any specialty, but in reality programs will likely resemble the specialties to whose programs they are administratively aligned, if for no other reason due to the normal give and take of academic medical center negotiations and politics.
A third major problem is the two-year time frame. While I do believe that most clinical informatics subspecialists are likely to require two years to complete their training, the arbitrary two-year time frame is at odds with the growing change in medical education from time-based tp competency-based education. Learning informatics is no more like "steeping tea" than medical education [1].
One of the outcomes from these problems is that it may be too difficult for there to be an adequate number of fellowship programs for qualified individuals from all medical specialties to make the subspecialty truly viable. If each fellowship has a one-off situation (i.e., only able or willing to take trainees from one or a small number of specialties) or must be funded, to use the words of my institution's chief medical officer, "creatively," this may undermine the long-term viability of the subspecialty.
Alternative Approach
I cannot criticize the proposed approach without offering an alternative, and I believe there are approaches that could be rigorous enough to ensure an equally if not more robust educational and training experience than the proposed fellowship model. It would no doubt test the boundaries of a tradition-bound organization like ACGME but could also show innovation reflective (and indeed required) of modern education.
We must remember that there will be three basic activities of clinical informatics subspecialty trainees:
- Clinical informatics education to master the core knowledge of the field
- Clinical informatics project work to gain skills and practical experience
- Clinical practice to maintain their skills in their primary medical specialty
Second, how would trainees get their practical hands-on project work? Again, many informatics programs, certainly ours, have developed mechanisms by which students can do internships or practicums in remote location through a combination of affiliation agreements, local mentoring, and remote supervision. While our program currently has students performing 3-6 months at a time of these, I see no reason why the practical experience could not be expanded to a year or longer. Strict guidelines for experience and both local and remote mentoring could be put in place to insure quality.
Finally, what about clinical practice? This may be easiest of all. Requiring a trainee to perform a certain volume of clinical practice, while adhering to all appropriate requirements for licensure and maintenance of certification, should be more than adequate to insure practice in their primary specialty. Many informatics distance learning students are already maintaining their clinical practices to maintain their livelihood. Making clinical practice explicit, instead of as something requiring supervision, will also allow training to be more financially viable for the fellow. Any costs of tuition and practical work could easily be offset by clinical practice revenue.
There would obviously need to be some sort of national infrastructure to set standards and monitor progress of clinical informatics trainees. There are any number of organizations that could perform this task, such as AMIA, and it could perhaps be a requirement of accreditation.
In fact, ACGME and the larger medical education community may learn from alternative approaches like this for training in other specialties. One major national concern these days is that number of residency positions for medical school graduates is not keeping up with the increases of medical school enrollment or, for that matter, the national need for physicians [3]. It is possible that alternative approaches like this could expand the capacity of all medical specialties and subspecialties, and not just clinical informatics.
References
1. Hodges BD, A tea-steeping or i-Doc model for medical education? Academic Medicine, 2010. 85: S34-S44.
2. Hersh WR, The full spectrum of biomedical informatics education at Oregon Health & Science University. Methods of Information in Medicine, 2007. 46: 80-83.
3. Iglehart JK, The residency mismatch. New England Journal of Medicine, 2013. 369: 297-299.
Wednesday, November 6, 2013
Evolution of Medical Education to Competency-Based Approaches: Can or Should Informatics Education Adopt Them?
There are major changes taking place in the education of physicians and other healthcare professionals. I have had the opportunity to be involved up close as OHSU undertakes a major transformation of its medical school curriculum.
As I work with our medical education leaders, and become familiar with the latest methods, jargon, and research, I ask myself whether informatics can or should adopt these approaches in our educational programs. In this posting I will review the methods and techniques being used in medical education, and in a subsequent posting will explore how these may be adapted by informatics education.
One of the major thrusts in medical education has been the move to competency-based medical education (CBME). A great deal of new jargon has emerged, the understanding of which is critical to read its literature [1].
Probably the biggest change being advocated is a transition from time-based to competency-based education [2]. This is based on the notion that our traditional time-based approach of students being "steeped" (like tea) in an educational program for a fixed period of time will result in their somehow emerging as competent practitioners. More recent thinking is that students acquire competencies at different rates from their fellow students, and there is no reason why all students should be forced into the same time period to attain them. This also has practical relevance in the need to increase the number of physicians in the US as the baby boomer population ages and more patients are brought into the healthcare system through healthcare reform.
Some aspects of these concepts are not new. For example, in the 1960s Bloom advanced the concept of "mastery learning" [3]. The idea behind mastery learning is that the variable changes from the traditional fixed time and measuring learning on a test to insuring that learning is complete and the variable being time, namely different time to master a subject for different students.
Another development in medical education in recent years has been the acceptance of a common taxonomy of competency domains [4]. This started with the six competency domains developed by the ACGME and ABMS, with 36 competencies delineated within them. More recently, the AAMC has added two additional domains and both added and refined the competencies to increase the total to 58.
An additional important development is the Next Accreditation System for graduate (post-medical school) medical education [5]. Central to this effort is the development of milestones (developmentally based, specialty-specific) that are achieved at five levels throughout medical school, residency, and advanced training or clinical practice. Seven specialties are launching milestones in 2013, with the remainder following in 2014 [6]. Training programs will be assessed based on the achievement of milestones by their trainees.
Another related concept to emerge is that of the entrustable professional activity (EPA), which is the notion that there are tasks or responsibilities that can be entrusted to a trainee once competence is achieved so that he or she can execute them without supervision [7]. EPA activities have been proposed for family medicine physicians [8], with others likely to follow.
These changes in medical education are also augmented by changes brought about by new technology. Two have been noted in particular: massive open online courses (MOOCs), which allow "flipped classrooms" and digital badges, both of which can create electronic means to validate milestones, EPAs, and other achievements [9]. Many other roles have been discerned as well [10-11].
Technology also foments other change in medical education. For example, the wedding of individuals to smartphones and tablets may have unintended consequences [12]. In addition, just as EHRs profoundly impact the workflow of physicians, they likewise impact the workflow of students [13]. Additional instruction may be required in reading and writing to the EHR [14] as well as its proper use in the patient-physician encounter [15-16]. Technology also impacts the role of the "master diagnostician," a teacher who also plays an outsized role in many academic medical centers [17].
The approach to educating physicians is clearly changing. The bulk of medical education leaders advocate an approach that changes the focus from one that is time-based to a more competency-based design. As such, there is no reason why medical education should last four years for everyone, and many aspects can be more individualized based on prior knowledge before medical school, learning systems, and other attributes of students.
It turns out that I have been using some of the above innovations in my own teaching, although I did not realize there were names for what I was doing. For example, in my introductory informatics course, taught in various venues as OHSU BMI 510 or AMIA 10x10, I sometimes teach the course "on campus." When I do this, it does not make sense to stand up and give lectures in the traditional sense. Instead, I use the "flipped classroom" approach of having students view the lectures before coming to class and then spending class time reviewing the material, answering questions, and discussing issues more deeply. (In online offerings of the class, these activities are carried out in discussion forums on our learning management system.)
In addition, for at least one class I teach, I use a variant of mastery learning. Another course I teach in our program is an elective course, BMI 536 - Evidence-Based Medicine. I believe that the best way to teach this topic is for students to repeatedly carry out the techniques of asking appropriate clinical questions and critically appraising the evidence until they get them right, i.e., have mastered them. They must do this before receiving a passing grade in the course.
In a future posting, I will explore whether this larger competency-based approach is appropriate to the education of informaticians, and if so, how we might implement it. I will also look at this from the standpoint of fellowships for the new physician subspecialty of clinical informatics.
References
1. Carraccio CL and Englander R, From Flexner to competencies: reflections on a decade and the journey ahead. Academic Medicine, 2013. 88: 1067-1073.
2. Hodges BD, A tea-steeping or i-Doc model for medical education? Academic Medicine, 2010. 85: S34-S44.
3. Kulik CC and Kulik JA, Effectivenss of mastery learning programs: a meta-analysis. Review of Educational Research, 1990. 60: 265-299.
4. Englander R, Cameron T, Ballard AJ, Dodge J, Bull J, and Aschenbrener CA, Toward a common taxonomy of competency domains for the health professions and competencies for physicians. Academic Medicine, 2013. 88: 1088-1094.
5. Nasca TJ, Philibert I, Brigham T, and Flynn TC, The next GME accreditation system--rationale and benefits. New England Journal of Medicine, 2012. 366: 1051-1056.
6. Swing SR, Beeson MS, Carraccio C, Coburn M, Iobst W, Selden NR, et al., Educational milestone development in the first 7 specialties to enter the next accreditation system. Journal of Graduate Medical Education, 2013. 5: 98-106.
7. Ten Cate O, Nuts and bolts of enstrustable professional activities. Journal of Graduate Medical Education, 2013. 5: 157-158.
8. Shaughnessy AF, Sparks J, Cohen-Osher M, Goodell KH, Sawin GL, and Gravel J, Entrustable professional activities in family medicine. Journal of Graduate Medical Education, 2013. 5: 112-118.
9. Mehta NB, Hull AL, Young JB, and Stoller JK, Just imagine: new paradigms for medical education. Academic Medicine, 2013. 88: 1418-1423.
10. Triola MM, Friedman E, Cimino C, Geyer EM, Wiederhorn J, and Mainiero C, Health information technology and the medical school curriculum. American Journal of Managed Care, 2010. 16(12 Suppl HIT): SP54-SP56.
11. Anonymous, Health Professions Education: Accelerating Innovation Through Technology. 2013, The Blue Ridge Academic Health Group: Atlanta GA, http://whsc.emory.edu/blueridge/publications/archive/blue-ridge-2013.pdf.
12. Wu R, Rise of the cyborgs: residents with smartphones, iPads, and Androids. Journal of Graduate Medical Education, 2013. 5: 161-162.
13. Ellaway RH, Graves L, and Greene PS, Medical education in an electronic health record-mediated world. Medical Teacher, 2013. 35: 282-286.
14. Han H and Lopp L, Writing and reading EHR documentation: an entirely new world. Medical Education Online, 2013. 18: 18634. http://med-ed-online.net/index.php/meo/article/view/18634.
15. Pearce C, Dwan K, Arnold M, Phillips C, and Trumble S, Doctor, patient and computer--a framework for the new consultation. International Journal of Medical Informatics, 2009. 78: 32-38.
16. Pearce C, Arnold M, Phillips C, Trumble S, and Dwan K, The patient and the computer in the primary care consultation. Journal of the American Medical Informatics Association, 2011. 18: 138-142.
17. Dhaliwal G and Detsky AS, The evolution of the master diagnostician. Journal of the American Medical Association, 2013. 310: 579-580.
As I work with our medical education leaders, and become familiar with the latest methods, jargon, and research, I ask myself whether informatics can or should adopt these approaches in our educational programs. In this posting I will review the methods and techniques being used in medical education, and in a subsequent posting will explore how these may be adapted by informatics education.
One of the major thrusts in medical education has been the move to competency-based medical education (CBME). A great deal of new jargon has emerged, the understanding of which is critical to read its literature [1].
Probably the biggest change being advocated is a transition from time-based to competency-based education [2]. This is based on the notion that our traditional time-based approach of students being "steeped" (like tea) in an educational program for a fixed period of time will result in their somehow emerging as competent practitioners. More recent thinking is that students acquire competencies at different rates from their fellow students, and there is no reason why all students should be forced into the same time period to attain them. This also has practical relevance in the need to increase the number of physicians in the US as the baby boomer population ages and more patients are brought into the healthcare system through healthcare reform.
Some aspects of these concepts are not new. For example, in the 1960s Bloom advanced the concept of "mastery learning" [3]. The idea behind mastery learning is that the variable changes from the traditional fixed time and measuring learning on a test to insuring that learning is complete and the variable being time, namely different time to master a subject for different students.
Another development in medical education in recent years has been the acceptance of a common taxonomy of competency domains [4]. This started with the six competency domains developed by the ACGME and ABMS, with 36 competencies delineated within them. More recently, the AAMC has added two additional domains and both added and refined the competencies to increase the total to 58.
An additional important development is the Next Accreditation System for graduate (post-medical school) medical education [5]. Central to this effort is the development of milestones (developmentally based, specialty-specific) that are achieved at five levels throughout medical school, residency, and advanced training or clinical practice. Seven specialties are launching milestones in 2013, with the remainder following in 2014 [6]. Training programs will be assessed based on the achievement of milestones by their trainees.
Another related concept to emerge is that of the entrustable professional activity (EPA), which is the notion that there are tasks or responsibilities that can be entrusted to a trainee once competence is achieved so that he or she can execute them without supervision [7]. EPA activities have been proposed for family medicine physicians [8], with others likely to follow.
These changes in medical education are also augmented by changes brought about by new technology. Two have been noted in particular: massive open online courses (MOOCs), which allow "flipped classrooms" and digital badges, both of which can create electronic means to validate milestones, EPAs, and other achievements [9]. Many other roles have been discerned as well [10-11].
Technology also foments other change in medical education. For example, the wedding of individuals to smartphones and tablets may have unintended consequences [12]. In addition, just as EHRs profoundly impact the workflow of physicians, they likewise impact the workflow of students [13]. Additional instruction may be required in reading and writing to the EHR [14] as well as its proper use in the patient-physician encounter [15-16]. Technology also impacts the role of the "master diagnostician," a teacher who also plays an outsized role in many academic medical centers [17].
The approach to educating physicians is clearly changing. The bulk of medical education leaders advocate an approach that changes the focus from one that is time-based to a more competency-based design. As such, there is no reason why medical education should last four years for everyone, and many aspects can be more individualized based on prior knowledge before medical school, learning systems, and other attributes of students.
It turns out that I have been using some of the above innovations in my own teaching, although I did not realize there were names for what I was doing. For example, in my introductory informatics course, taught in various venues as OHSU BMI 510 or AMIA 10x10, I sometimes teach the course "on campus." When I do this, it does not make sense to stand up and give lectures in the traditional sense. Instead, I use the "flipped classroom" approach of having students view the lectures before coming to class and then spending class time reviewing the material, answering questions, and discussing issues more deeply. (In online offerings of the class, these activities are carried out in discussion forums on our learning management system.)
In addition, for at least one class I teach, I use a variant of mastery learning. Another course I teach in our program is an elective course, BMI 536 - Evidence-Based Medicine. I believe that the best way to teach this topic is for students to repeatedly carry out the techniques of asking appropriate clinical questions and critically appraising the evidence until they get them right, i.e., have mastered them. They must do this before receiving a passing grade in the course.
In a future posting, I will explore whether this larger competency-based approach is appropriate to the education of informaticians, and if so, how we might implement it. I will also look at this from the standpoint of fellowships for the new physician subspecialty of clinical informatics.
References
1. Carraccio CL and Englander R, From Flexner to competencies: reflections on a decade and the journey ahead. Academic Medicine, 2013. 88: 1067-1073.
2. Hodges BD, A tea-steeping or i-Doc model for medical education? Academic Medicine, 2010. 85: S34-S44.
3. Kulik CC and Kulik JA, Effectivenss of mastery learning programs: a meta-analysis. Review of Educational Research, 1990. 60: 265-299.
4. Englander R, Cameron T, Ballard AJ, Dodge J, Bull J, and Aschenbrener CA, Toward a common taxonomy of competency domains for the health professions and competencies for physicians. Academic Medicine, 2013. 88: 1088-1094.
5. Nasca TJ, Philibert I, Brigham T, and Flynn TC, The next GME accreditation system--rationale and benefits. New England Journal of Medicine, 2012. 366: 1051-1056.
6. Swing SR, Beeson MS, Carraccio C, Coburn M, Iobst W, Selden NR, et al., Educational milestone development in the first 7 specialties to enter the next accreditation system. Journal of Graduate Medical Education, 2013. 5: 98-106.
7. Ten Cate O, Nuts and bolts of enstrustable professional activities. Journal of Graduate Medical Education, 2013. 5: 157-158.
8. Shaughnessy AF, Sparks J, Cohen-Osher M, Goodell KH, Sawin GL, and Gravel J, Entrustable professional activities in family medicine. Journal of Graduate Medical Education, 2013. 5: 112-118.
9. Mehta NB, Hull AL, Young JB, and Stoller JK, Just imagine: new paradigms for medical education. Academic Medicine, 2013. 88: 1418-1423.
10. Triola MM, Friedman E, Cimino C, Geyer EM, Wiederhorn J, and Mainiero C, Health information technology and the medical school curriculum. American Journal of Managed Care, 2010. 16(12 Suppl HIT): SP54-SP56.
11. Anonymous, Health Professions Education: Accelerating Innovation Through Technology. 2013, The Blue Ridge Academic Health Group: Atlanta GA, http://whsc.emory.edu/blueridge/publications/archive/blue-ridge-2013.pdf.
12. Wu R, Rise of the cyborgs: residents with smartphones, iPads, and Androids. Journal of Graduate Medical Education, 2013. 5: 161-162.
13. Ellaway RH, Graves L, and Greene PS, Medical education in an electronic health record-mediated world. Medical Teacher, 2013. 35: 282-286.
14. Han H and Lopp L, Writing and reading EHR documentation: an entirely new world. Medical Education Online, 2013. 18: 18634. http://med-ed-online.net/index.php/meo/article/view/18634.
15. Pearce C, Dwan K, Arnold M, Phillips C, and Trumble S, Doctor, patient and computer--a framework for the new consultation. International Journal of Medical Informatics, 2009. 78: 32-38.
16. Pearce C, Arnold M, Phillips C, Trumble S, and Dwan K, The patient and the computer in the primary care consultation. Journal of the American Medical Informatics Association, 2011. 18: 138-142.
17. Dhaliwal G and Detsky AS, The evolution of the master diagnostician. Journal of the American Medical Association, 2013. 310: 579-580.
Monday, November 4, 2013
Will MOOCs Cause Disruptive Innovation in Higher Education or Just Be a Valuable Resource?
As one who has been involved in online learning for nearly a decade and a half, I have been following the development of massive open online courses (MOOCs) with great interest. As someone who struggles daily with managing an educational program that is mandated by my university to be financially self-sufficient, I take great interest in financial models for educational programs and the new challenges these disruptive technologies may bring.
MOOCs have been around for a couple years now, and they have even started to permeate into the biomedical and health informatics world. I am aware of at least two informatics-related MOOCs (one from Georgia Institute of Technology and the other from University of Minnesota) as well as another that focuses on healthcare analytics. In addition, a colleague from Australia has been fashioning the ONC health IT curriculum into one giant MOOC, although it is not a course in the sense of one registering for it and having criteria for successfully completing it.
We are also starting to see some research findings about MOOCs. A first analysis was recently published about a MOOC from Massachusetts Institute of Technology (MIT) [1]. The course studied was 6.002x, an introductory course in circuits and electronics offered through edX, one of the two large US-based consortia offering MOOCs. Typical of MOOCs, about 155,000 people registered for the course and about 7000 completed it. This is still an impressive number completing the course, and the research noted factors that kept students engaged and successful. One of the most prominent factors was interaction with fellow students in the non-required online discussion forums. The researchers also determined that students spent the most time in online lectures when learning materials but most often referred to the online textbook when completing exams and browsed the discussion forums for help in completing homework assignments.
Other research in the same issue of the journal where this paper was published looked at handling some of the challenges of large-scale online education, such as grading of essay materials [2] and preventing cheating in these types of courses [3]. One approach to essay grading, highly controversial, is automated grading that uses machine learning approaches, while another approach uses "calibrated peer review" among participating students.
At this point in time, MOOCs have not yet led to true disruptive innovation in an industry (higher education) that has maintained resistance to such innovation, although the originator of the concept of disruptive innovation believes this will happen soon [4]. Perhaps the development that comes closest to disrupting higher education is the launching of a $6600 master's degree in computer science by Georgia Institute of Technology [5]. This program does not replace the institution's residential $40,000 master's program. But if successful, it will demonstrate a possible pathway to high-quality higher education that is significantly less expensive than conventionally delivered education.
In our OHSU graduate program in biomedical informatics, we have found that distance learning's attributes are more about the flexibility and the reach of our educational program across the planet than lowered cost. We consider it important for us to still provide the value of a comprehensive higher education program, which includes:
There is no question that online education delivery will continue to grow, and soon start to permeate highly resistant fields, such as medicine [6]. As with many technology-related endeavors, I believe that the most likely models to emerge will be hybrid models, i.e., those that make use of resources like MOOCs but still offer comprehensive educational experiences. I can easily see institutions of higher education licensing or otherwise using MOOCs in their educational offerings, with the institution filling in the additional value required for a complete education.
These benefits of MOOCs will have the potential to lower costs and introduce efficiencies, but probably not to the extent of widespread sub-$10,000 master's degrees. There will be other value, however, such as the reach and flexibility of online learning. Perhaps the best of all worlds will allow higher education to focus on other activities it can perform well, such as personal and career development as well as exposure to "real world" work environments.
References
1. Breslow L, Pritchard DE, DeBoer J, Stump GS, Ho AD, and Seaton DT, Studying learning in the worldwide classroom: research into edX’s first MOOC. Research and Practice in Assessment, 2013. 8. http://www.rpajournal.com/studying-learning-in-the-worldwide-classroom-research-into-edxs-first-mooc/.
2. Balfour SP, Assessing writing in MOOCs: automated essay scoring and calibrated peer review. Research and Practice in Assessment, 2013. 8. http://www.rpajournal.com/assessing-writing-in-moocs-automated-essay-scoring-and-calibrated-peer-review/.
3. Meyer JP and Zhu S, Fair and equitable measurement of student learning in MOOCs: an introduction to item response theory, scale linking, and score equating. Research and Practice in Assessment, 2013. 8. http://www.rpajournal.com/fair-and-equitable-measurement-of-student-learning-in-moocs-an-introduction-to-item-response-theory-scale-linking-and-score-equating/.
4. Christensen CM and Horn MB, Innovation Imperative: Change Everything, New York Times. November 1, 2013. http://www.nytimes.com/2013/11/03/education/edlife/online-education-as-an-agent-of-transformation.html.
5. Lewin T, Master’s Degree Is New Frontier of Study Online, New York Times. August 17, 2013. http://www.nytimes.com/2013/08/18/education/masters-degree-is-new-frontier-of-study-online.html.
6. Mehta NB, Hull AL, Young JB, and Stoller JK, Just imagine: new paradigms for medical education. Academic Medicine, 2013. 88: 1418-1423.
MOOCs have been around for a couple years now, and they have even started to permeate into the biomedical and health informatics world. I am aware of at least two informatics-related MOOCs (one from Georgia Institute of Technology and the other from University of Minnesota) as well as another that focuses on healthcare analytics. In addition, a colleague from Australia has been fashioning the ONC health IT curriculum into one giant MOOC, although it is not a course in the sense of one registering for it and having criteria for successfully completing it.
We are also starting to see some research findings about MOOCs. A first analysis was recently published about a MOOC from Massachusetts Institute of Technology (MIT) [1]. The course studied was 6.002x, an introductory course in circuits and electronics offered through edX, one of the two large US-based consortia offering MOOCs. Typical of MOOCs, about 155,000 people registered for the course and about 7000 completed it. This is still an impressive number completing the course, and the research noted factors that kept students engaged and successful. One of the most prominent factors was interaction with fellow students in the non-required online discussion forums. The researchers also determined that students spent the most time in online lectures when learning materials but most often referred to the online textbook when completing exams and browsed the discussion forums for help in completing homework assignments.
Other research in the same issue of the journal where this paper was published looked at handling some of the challenges of large-scale online education, such as grading of essay materials [2] and preventing cheating in these types of courses [3]. One approach to essay grading, highly controversial, is automated grading that uses machine learning approaches, while another approach uses "calibrated peer review" among participating students.
At this point in time, MOOCs have not yet led to true disruptive innovation in an industry (higher education) that has maintained resistance to such innovation, although the originator of the concept of disruptive innovation believes this will happen soon [4]. Perhaps the development that comes closest to disrupting higher education is the launching of a $6600 master's degree in computer science by Georgia Institute of Technology [5]. This program does not replace the institution's residential $40,000 master's program. But if successful, it will demonstrate a possible pathway to high-quality higher education that is significantly less expensive than conventionally delivered education.
In our OHSU graduate program in biomedical informatics, we have found that distance learning's attributes are more about the flexibility and the reach of our educational program across the planet than lowered cost. We consider it important for us to still provide the value of a comprehensive higher education program, which includes:
- An up-to-date curriculum based on a solid foundation
- Faculty who are international leaders in research and practice
- Ability to find and carry out an internship or practicum experience
- Career development and advising
- Connections to industry and others in the field
There is no question that online education delivery will continue to grow, and soon start to permeate highly resistant fields, such as medicine [6]. As with many technology-related endeavors, I believe that the most likely models to emerge will be hybrid models, i.e., those that make use of resources like MOOCs but still offer comprehensive educational experiences. I can easily see institutions of higher education licensing or otherwise using MOOCs in their educational offerings, with the institution filling in the additional value required for a complete education.
These benefits of MOOCs will have the potential to lower costs and introduce efficiencies, but probably not to the extent of widespread sub-$10,000 master's degrees. There will be other value, however, such as the reach and flexibility of online learning. Perhaps the best of all worlds will allow higher education to focus on other activities it can perform well, such as personal and career development as well as exposure to "real world" work environments.
References
1. Breslow L, Pritchard DE, DeBoer J, Stump GS, Ho AD, and Seaton DT, Studying learning in the worldwide classroom: research into edX’s first MOOC. Research and Practice in Assessment, 2013. 8. http://www.rpajournal.com/studying-learning-in-the-worldwide-classroom-research-into-edxs-first-mooc/.
2. Balfour SP, Assessing writing in MOOCs: automated essay scoring and calibrated peer review. Research and Practice in Assessment, 2013. 8. http://www.rpajournal.com/assessing-writing-in-moocs-automated-essay-scoring-and-calibrated-peer-review/.
3. Meyer JP and Zhu S, Fair and equitable measurement of student learning in MOOCs: an introduction to item response theory, scale linking, and score equating. Research and Practice in Assessment, 2013. 8. http://www.rpajournal.com/fair-and-equitable-measurement-of-student-learning-in-moocs-an-introduction-to-item-response-theory-scale-linking-and-score-equating/.
4. Christensen CM and Horn MB, Innovation Imperative: Change Everything, New York Times. November 1, 2013. http://www.nytimes.com/2013/11/03/education/edlife/online-education-as-an-agent-of-transformation.html.
5. Lewin T, Master’s Degree Is New Frontier of Study Online, New York Times. August 17, 2013. http://www.nytimes.com/2013/08/18/education/masters-degree-is-new-frontier-of-study-online.html.
6. Mehta NB, Hull AL, Young JB, and Stoller JK, Just imagine: new paradigms for medical education. Academic Medicine, 2013. 88: 1418-1423.