Discerning the Evidence Base for Health Information Exchange (HIE)

Some of the most important projects in my career have come out of nowhere, almost by coincidence, but turning into major commitments as well as important endeavors for the field. I am about to begin a new project that originated in a similarly unexpected manner.

This project came about because our Department of Medical Informatics & Clinical Epidemiology (DMICE) is one of the 12 Evidence-Based Practice Centers (EPCs) funded by the Agency for Healthcare Research and Quality (AHRQ). Being an EPC gives these centers the opportunity to bid to perform "task orders," which mainly consist of "evidence reports" that usually consist of one or more systematic reviews and sometimes additional analysis. DMICE houses the Pacific Northwest EPC, which is a major activity of the "CE" part of DMICE.

Most of the AHRQ evidence reports focus on clinical topics, such as traumatic brain injury or myocardial infarction. AHRQ has commissioned a few reports on informatics-related topics over the years, such as telemedicine and health information technology. (I was principal investigator [PI] of the telemedicine reports it commissioned, which included an initial analysis focused on Medicare patients [1-2], a supplementary analysis on all patients, and an update [3-4].)

A couple months ago, the Director of our EPC informed me that AHRQ had released a "request for task order" (RFTO) seeking bids to prepare an evidence report on health information exchange (HIE). Needless to say, I jumped at the chance to serve as PI and prepare a proposal. I was then delighted to receive word in mid-September that our proposal had been selected for funding. We have a great team that combines informatics and evidence-based medicine expertise.

This 18-month project will become one of my major projects going forward. The timing could actually not be better, with the winding down of my big Office of the National Coordinator for Health IT (ONC) projects.

I am looking forward to undertaking this project. HIE is one of the most important activities of informatics now. The benefit of a standards-based, interoperable health IT ecosystem will not be realized unless, to quote former AHRQ Director Dr. Carolyn Clancy, "data follows the patient" wherever they go in the healthcare system. Or, as started by Dr. William Yasnoff, former Senior Federal Advisor for the National Health Information Infrastructure, there must be "anytime, anywhere access to data for patient care." Or even, as noted more recently by Dr. John Halamka, ACO = HIE + analytics. HIE has a growing accumulation of evidence concerning its effectiveness, obstacles, and sustainability.

We will follow the usual EPC approach in producing the evidence report. Our first activity will be to develop a set of key questions that the report will answer, embedded into a conceptual framework that shows the interrelationships among the questions. This process will be aided by engaging a set of "key informants" who represent various stakeholder groups, such as patients, providers, healthcare organizations, health policymakers, and vendors. They will help us in a process of "topic refinement" to specify the key questions [5].

Once our key questions are finalized, we will then perform a systematic review on each one. Systematic reviews differ from other kinds of topic reviews in that the questions are focused and amenable to being answered with scientific evidence [6]. They not only provide an inventory of existing scientific knowledge on a given topic but also inform us of the gaps in our knowledge.

Systematic reviews always begin with a comprehensive literature search, which casts a wide net for all possible evidence. The retrieved titles and abstracts are read and analyzed to determine which papers might meet inclusion criteria and should be pulled for reading of their full text. Those that meet the inclusion criteria are abstracted for the evidence, which is then is entered into evidence tables. If enough evidence from similar studies is retrieved, then a meta-analysis might be performed, where the data from each study is aggregated as if it were a single study. Meta-analysis summarizes the evidence as well as increases its statistical power.

This report will not focus solely on the evidence for the benefit of HIE. While that will clearly be an important part, we will also look at other aspects of HIE.

The first key question will likely concern the effectiveness of HIE. This will be assessed based on a variety of outcomes of its use, including healthcare process outcomes, intermediate outcomes, economic outcomes, clinical outcomes, and population-level outcomes. Other key questions will likely focus on:

• Usage
• Usability
• Harms
• Facilitators and barriers to successful use
• Sustainability

We will also look at attributes of HIE that may influence success, such as the setting of HIE use, patient population(s), and others.

One of the challenges for defining evidence for HIE is that it is used as a means to other ends, usually improved clinical care. Since many other factors may influence care, it may be hard to tease out the true value of HIE. But we will aggregate all the evidence we can find to best discern the scientific picture of its value and limitations.

References

1. Hersh, WR, Helfand, M, et al. (2001). Clinical outcomes resulting from telemedicine interventions: a systematic review. BMC Medical Informatics and Decision Making. 1: 5. http://www.biomedcentral.com/1472-6947/1/5.
2. Hersh, W, Helfand, M, et al. (2002). A systematic review of the efficacy of telemedicine for making diagnostic and management decisions. Journal of Telemedicine and Telecare. 8: 197-209.
3. Hersh, WR, Hickam, DH, et al. (2006). The evidence base of telemedicine: overview of the supplement. Journal of Telemedicine & Telecare. 12(Supp 2): 1-2.
4. Hersh, WR, Hickam, DH, et al. (2006). Diagnosis, access, and outcomes: update of a systematic review on telemedicine services. Journal of Telemedicine & Telecare. 12(Supp 2): 3-31.

5. Buckley, DI, Ansari, M, et al. (2013). The Refinement of Topics for Systematic Reviews: Lessons and Recommendations From the Effective Health Care Program. Rockville, MD, Agency for Healthcare Quality & Research.

6. Khan, K, Kunz, R, et al. (2011). Systematic Reviews to Support Evidence-Based Medicine, 2nd Edition. Boca Raton, FL, CRC Press.

Gimme Some Analytics (We Already Have It!)

One of the most common requests we hear for new curricular content in our clinical informatics graduate program curriculum is for a course on "analytics." Indeed, this word has officially achieved buzzword status in informatics and maybe even all of healthcare. This was evidenced recently by the OHSU School of Medicine Senior Associate Dean for Education speaking with a hospital CEO in our state about what skills were most important in training our future physicians. The CEO replied with one: "predictive analytics." I am not sure I would value that skill highest in a physician I was seeing for a medical problem, although I would appreciate that physician having been trained in a data-driven, system-oriented approach.

I wrote in this blog last year that the focus of work of clinical informatics itself will likely change from electronic health record implementation to analytics, i.e., optimizing the systems we now have so all of the data accumulating in them can be used to improve health and healthcare delivery. I have also taken, like many other informatics researchers, a great interest in data-related issues [1, 2].

By the same token, I believe we need to be careful about the hype around analytics, something I alluded to in a recent posting reminding data scientists that they also needed to be research methodology scientists. Since that time, a colleague provided some additional examples of this, noting a recent talk she attended by a CEO of a data analytics company who proudly declared that he did not need to know anything about the underlying domain of medicine; he was only a data scientist. He proceeded to describe some correlations that his product had recently uncovered, such as a diagnosis of chronic kidney disease being associated with an ateriovenous (AV) fistula and septicemia being associated with hospitalization. (To which the medically knowledgeable people replied, "Duh!") In other words, finding correlations among things we already know clinically is hardly an advance for analytics. This is also borne out in another recent paper on "Why big won't cure us," with the author pointing out a myriad of technical, financial, and ethical issues that must be addressed before we will be able to make use of big data routinely in clinical practice [3].

Another issue is that while the use of data analytics is important and will grow, we cannot say with certainty what proportion of informatics professionals will actually be working in this area. There is no question that most informatics professionals will be working with data and information in some way, as these are the "lifeblood" of healthcare [4]. We also know that for every person who explicitly "does" analytics, there are some number of people, probably a larger proportion, who are either supporting analytics or doing other health IT work, whether with EHRs or other technologies. This is borne out by a recent report from the McKinsey consulting firm, which forecasts a demand for 140,000 to 190,000 positions for all data scientists in all fields (not limited to healthcare), and also another need for an addition 1.5 million managers and analysts who "can ask the right questions and consume the results of the analysis of big data effectively" [4].

What our informatics educational program needs to teach was borne out in a recent conversation with Brian Sikora, who is Director of Data & Information Management Enhancement in the Kaiser-Permanente Northwest Region. His service currently employs 110 analysts and plans to add more. Also good news for our students was our discussion about possible internship opportunities.

Mr. Sikora described four types of analysts in his department to our career development specialist, Ms. Virginia Lankes:

• Data analysts - the most technical group who do the architecture
• Report analysts – design reports, dashboards, key performance indicators; wider background, business intelligence domain.
• Business systems analysts – understand system, workflow and the people they are working with to process and know how data is produced in a specific area. “Here’s the problem - help us solve it.” Deep expertise in operational areas.
• Informatics analysts – (the largest group) which includes statisticians; they help operations leaders interpret and analyze data. Data mining, text mining.

Mr. Sikora listed key skills required of analysts joining his team:

1. Programming skills - analytics professionals must have programming skills, especially in data-related areas for locating and extracting data, using tools such as SQL and SAS.
2. Statistics
3. Understanding the healthcare environment
4. Communication skills - ability to work with clinical, administrative, and financial staff to understand their programs and present solutions in written and oral form
5. Critical thinking - including the ability to understand a business problem, identify the appropriate data elements, extract and aggregate the data, and use it to solve the practical problem

One reassuring aspect of this list is that it parallels the "domains" that we use in the core curriculum of our graduate (master's and PhD) program that has existed for almost 20 years. Our program has always required courses in the domains of biomedical informatics, computer science, healthcare, organization behavior and management, and evaluation (including statistics). We also have a course in scientific writing and communication. Thus our curriculum already has the essential basics of "analytics," even if we do not call it by that name. Clearly we need to make this more explicit, and organize some structure that reassures students and their potential employers that they are indeed well-trained in that area. At some point we may add a course with the name "analytics" explicitly in the title, but it will need to be a course that synergizes work in programming, statistics, and problem-solving, and not one that is standalone in nature.

The jury is clearly out on who and how many will be doing what in analytics in the future work of informatics and larger healthcare. Just as I called for research delineating the informatics workforce years ago [5], we need a similar analysis with regards to analytics. We need to answer questions such as, how many people do we need to do the work of analytics, how many do we need to support it, what fraction of the informatics field will be working explicitly in analytics, and what training is needed for those who work directly in it and those who need to know it as part of a well-rounded informatics education.

References

1. Hersh, WR, Weiner, MG, et al. (2013). Caveats for the use of operational electronic health record data in comparative effectiveness research. Medical Care. 51(Suppl 3): S30-S37.
2. Hersh, WR, Cimino, JJ, et al. (2013). Recommendations for the use of operational electronic health record data in comparative effectiveness research. eGEMs (Generating Evidence & Methods to improve patient outcomes). 1: 14. http://repository.academyhealth.org/egems/vol1/iss1/14/.
3. Neff, G (2013). Why big data won't cure us. Big Data. 1: 117-123.
4. Manyika, J, Chui, M, et al. (2011). Big data: The next frontier for innovation, competition, and productivity, McKinsey Global Institute. http://www.mckinsey.com/insights/business_technology/big_data_the_next_frontier_for_innovation.

5. Hersh, WR (2006). Who are the informaticians? What we know and should know. Journal of the American Medical Informatics Association. 13: 166-170.

Further Evidence That Health IT Job Growth Has Been Underestimated, and Some Ramifications

New data from an analysis of online job postings confirms that employment growth in health information technology (HIT) has even further exceeded projections, driven by funding from the Health Information Technology for Economic and Clinical Health (HITECH) Act. In addition, two other reports show that the job market for those working with electronic health record (EHR) and related systems continues to be strong for employees and challenging for employers.

In the new analysis, Schwartz and colleagues used a comprehensive database of 84 million online job postings,  extracted out those related to HIT, and built a model aiming to determine the influence of HITECH [1]. The authors limited their focus to jobs that would be defined in the realm of clinical informatics. Although this was important for their goal of assessing the influence of HITECH, from the perspective of a biomedical informatics graduate program director like myself, this excluded other important informatics jobs, such as those in imaging informatics, bioinformatics, clinical research informatics, and other areas where a graduates of our program are employed.

The analysis classified jobs into two broad categories, HIT core jobs and HIT clinical user jobs. The former category included those developing, implementing, supporting, and selling EHRs, while the latter included clinicians, receptionists, technicians, and other personnel making heavy use of EHRs in their jobs. Wearing my informatics educational program director's hat, I was most interested in the authors' results for the HIT core job listings, as these individuals would be most likely to be employed in (and to seek education preparing them for) informatics careers.

Schwartz et al. counted a total of HIT-related 434,282 job postings between 2007-2011, with 226,356 HIT core jobs and 207,926 HIT-related clinical user jobs. Yes, not 41,000 [2], not even 51,000 [3], but over 226,000!

For both categories of HIT jobs combined, the authors categorized employer type and provided a percentage of all HIT jobs for each. The largest employer category was IT vendor, which included IT service providers, consulting firms, sales firms, and IT staffing firms hiring developers and posted 42% of all HIT-related listings. Another 39% were posted by healthcare provider organizations. The remaining 19% were either ambiguous as to the employer type (15%) or another type of employer (4%).

For all HIT jobs, they also categorized and tallied job responsibilities, which could be assigned to more than one category for a given posting. The most frequent job responsibility was implementation support, with 43% of jobs including responsibilities such as system installation, customization, building, debugging, purchasing, or workflow redesign. The next highest category was user training (27%), followed by system development (22%). This was followed by technical support, with 21% of jobs including the maintaining of continued technical functionality or providing customer support. Other responsibilities included IT strategy (long-term IT planning and system optimization in the clinical setting - 13%), sales (11%), and research (quantitative hypothesis testing using health IT systems - 6%).

Their model estimated that about 48% of the job growth was due to HITECH, with the remainder due to growth that would have continued at historical trends prior to HITECH. Some other interesting findings for the time period between 2007-2011 included HIT jobs growing from 0.75% to nearly 2.5% of all healthcare job postings (consistent with prior findings of healthcare organizations hiring one IT employee per 48-60 non-IT employees [2]) and an approximately four-fold increase in the number of jobs posted.

It will remain to be seen how strong the job market remains as the HITECH incentives wind down, although HIT will continue to be a cost of doing business in healthcare, especially as the system moves to payment models that require better management and use of data, such as accountable care organizations and primary care/patient-centered medical homes [4].

Two other reports show that healthcare organizations are responding to the need for hiring and maintaining more HIT talent. A report by Towers Watson that surveyed 100 healthcare organizations [5] found that 67% of reported problems in attracting experienced IT employees, with an even higher proportion (73%) finding problems hiring Epic-certified employees. These organizations also reported problems in retaining experienced IT employees (38%) and Epic-certified employees (52%) as well. Many fewer organizations reported problems in attracting (14%) or retaining (9%) new graduates with IT skills, indicating a strong incentive for those early in their careers to get up to speed quickly.

The Towers Watson report also discerned differences between employees and employers in the drivers of attracting and retaining IT personnel. For attracting personnel, employees listed job security and salary as their primary concerns, whereas employers believed that challenging work and organizational reputation were key drivers. There was more concordance on drivers of IT employee retention, with both employees and employers ranking salary and opportunities for career advancement highest.

The second report was the first of what will be an annual survey of the HIT workforce by HIMSS Analytics [6]. The survey was completed by 224 individuals who were HIMSS corporate members as well as hospital and health system IT executives. About three-quarters of respondents were employed by healthcare provider organizations, including standalone hospitals (45%), hospitals as part of an integrated delivery system (17%), and corporate offices of a delivery system (13%). The remaining respondents worked vendor organizations, including companies (20%) and consulting firms (4%).

Over 80% of healthcare provider organizations reported adding IT FTE in the past year, with half hiring 1-5 FTE and the rest hiring more. Only 8% reporting laying off staff. The most common areas for hiring were clinical application support (51% of all healthcare provider organizations hiring), help desk (51%), IT management (29%), financial application support (28%), system design and implementation (24%), IT security (22%), project management (21%), clinical informatics/clinical champion (19%), system integration (19%), user training (15%). About three-quarters of provider organizations outsourced rather than hired some of the above types of personnel. Essentially similar percentages were seen in hiring plans for the coming year.

A similar picture was seen for vendors and consultants for the past year, with over 90% adding FTE, and nearly 60% hiring more than 20 FTE. Most common areas for hiring included sale and marketing (88%), field support staff (84%), support staff (68%), and executive team (58%). Nearly a third reported laying off some staff. Similar to provider organizations, comparable percentages were noted in hiring plans for the coming year.

Various types of certification were deemed important, more so vendors than healthcare provider organizations. The most highly rated certifications were security professional, network/architecture support, database administrator, project manager, and informatics professional.

Strategies for retaining qualified staff were also important for providers and vendors, including offering professional development activities (60% for providers, 64% for vendors), paid tuition (48%, 41%), membership payment in professional associations (35%, 41%), and telecommuting (29%, 52%).

About 80% of providers and 57% of vendors reported lack of fully qualified staff as a barrier to achieving organizational IT goals. The most common reason for lack of staff was lack of qualified staff in their local region (43%, 56%). Both types of organizations reported hires being attracted to other organizations by more lucrative offers (25%, 19%). About 31% of provider organizations reported putting an IT initiative on hold due to inadequate staffing, with another 19% contemplating doing so.

Overall, these new data show that the HIT job market is robust, and provides great opportunities for a career. Of course, like essentially all professional areas these days, the job market will change. This will be driven in HIT by the tapering of the incentive funding from HITECH and the shift from implementing to optimizing (i.e,, making use of the data) systems. This will require that the skill sets of professionals, and the curricular content of educational programs, adapt to (or ideally anticipate) the likely changes.

References

1. Schwartz, A, Magoulas, R, et al. (2013). Tracking labor demand with online job postings: the case of health IT workers and the HITECH Act. Industrial Relations: A Journal of Economy and Society. 52: 941–968.
2. Hersh, W (2010). The health information technology workforce: estimations of demands and a framework for requirements. Applied Clinical Informatics. 1: 197-212.
3. Conn, J (2010). 50,000 new health IT workers might be needed. Modern Healthcare, May 25, 2010. http://www.modernhealthcare.com/apps/pbcs.dll/article?AID=/20100525/NEWS/100529949/.
4. Anonymous (2011). Support for Accountable Care: Recommended Health IT Infrastructure. Washington, DC, eHealth Initiative. http://www.ehidc.org/resource-center/reports/view_document/103-reports-support-for-accountable-care-recommended-health-it-infrastructure-accountable-care.
5. Anonymous (2013). Closing the IT Talent Gap in Health Care - The Towers Watson 2013 Health Care IT Survey Report, Towers Watson. http://www.towerswatson.com/en/Insights/IC-Types/Survey-Research-Results/2013/03/Closing-the-IT-Talent-Gap-in-Health-Care.
6. Anonymous (2013). 2013 HIMSS Workforce Survey. Chicago, IL, HIMSS Analytics. http://www.himssanalytics.com/research/AssetDetail.aspx?pubid=82097&tid=128.

Accolades for the Informatics Professor - Summer 2013 Edition

Periodically I like to share some of the accolades I receive in my work, and this posting is another installment.

First, I was featured in a piece in Health Data Management concerning my accomplishments generally and related to the HITECH program specifically.

Next, this blog was named among the best health information technology (HIT) blogs from an HIT recruiter site.

Finally, I had the opportunity to present in an interesting format at the Ignite Health meeting in Portland, OR on September 19, 2013. The format for each presenter was to have five minutes to talk and each slide advancing automatically after 15 seconds, for a total of 20 slides. It was fun by challenging, and I also had to go after a speaker who had a wonderfully choreographed presentation with beautiful graphical slides. It was quite challenging for an academic used to a longer time to talk and less graphical slides, but I made the best of it. I chose to focus my presentation and its slides on the new Informatics Discovery Lab in our department. There is also an index to the overall program.

Postscript: The day after posting, I was mentioned in this nice article on the clinical informatics board exam.