In This Issue
Frontiers of Engineering
December 15, 2009 Volume 39 Issue 4
Vol 39 No. 4 Winter 2009 Bridge on Frontiers of Engineering

Why Health Information Technology Doesn't Work

Friday, January 15, 2010

Author: Elmer V. Bernstam and Todd R. Johnson

To improve the quality of our health care while lowering its cost, we will make the immediate investments necessary to ensure that within five years all of America’s medical records are computerized. This will cut waste, eliminate red tape, and reduce the need to repeat expensive medical tests… it will save lives by reducing the deadly but preventable medical errors that pervade our health care system.

—Barack Obama
George Mason University, January 8, 2009


Widespread dissatisfaction with health care in America and rapid advancements in information technology have focused attention on information technology, which has dramatically improved efficiency and safety in other industries, as an obvious part of the solution to our health care woes. However, there is increasing evidence that the adoption of health information technology (HIT) will not guarantee comparable benefits in health care.

In fact, unmitigated enthusiasm for HIT may even be dangerous. Similar enthusiasm has repeatedly threatened the field of artificial intelligence (AI), resulting in cycles of excitement and disappointment (referred to as “AI winters”). Motivated by a desire to avoid “HIT winters,” we will briefly review the effects of HIT and the “semantic gap,” that is, the difference between “health data” and “health information.” In addition, we identify significant social and administrative barriers to the adoption of HIT in the context of the technical issues; because HIT is embedded in a social context, these technical issues must be resolved in a socially and administratively acceptable way. We conclude with research challenges that must be addressed before the full promise of HIT can be realized.

Effects of Health Information Technology

HIT is an “easy sell” to an American public increasingly dissatisfied with the U.S. health care system. Indeed, based on some evidence that HIT can improve the quality of health care (Chaudhry et al., 2006), prevent medical errors (Bates et al., 2001), and increase efficiency (Chaudhry et al., 2006), there seem to be some good reasons for optimism.

Unfortunately, many, perhaps most, HIT projects have failed (Littlejohns et al., 2003), and evidence shows that HIT can worsen health care quality in some ways by increasing errors (Koppel et al., 2005; Levenson and Turner, 1993), decreasing efficiency, and perhaps even increasing mortality (Han et al., 2005). The term “e-iatrogenesis” has been coined to describe the unintended deleterious consequences of HIT (Weiner et al., 2007).

Enough negative evidence has accumulated to prompt the Joint Commission (formerly the Joint Commission on Accreditation of Healthcare Organizations) to issue a “Sentinel Event Alert” (defined as “unexpected occurrence[s] involving death or serious physical or psychological injury, or the risk thereof”) cautioning health care organizations about potential hazards associated with the implementation and use of HIT (Joint Commission, 2008).

We’ve Been There Before: AI Winters

During the 1950s, we were faced with a different problem—the cold war. At that time, the government considered IT a promising solution (at least a partial solution) to the problem of tracking Russian communications. It was thought that if researchers could develop automated translation, we would be able to monitor Russian communications and scientific reports in “real time.” There was a great deal of optimism about this, and there were “…many predictions of fully automatic systems operating within a few years” (Hutchins, 2006).

Although many promising applications were found for the poor-quality automated translations that resulted, the optimistic predictions were not realized. To this day, the fundamental problem of context and meaning remains unsolved, making disambiguation difficult and resulting in some amusing failures. Anecdotal examples include: “The spirit is willing, but the flesh is weak” was translated from English → Russian → English as “The vodka is good, but the meat is rotten,” and “out of sight, out of mind” came out as “blind idiot.”

In 1966, the influential Automatic Language Processing Advisory Committee (ALPAC) concluded that “there is no immediate or predictable prospect of useful machine translation” (NRC, 1966). As a result, research funding was stopped, and little research was done on automated translation in the United States from 1967 to 1976, when it was revived and supported until 1989 (Hutchins, 2006). Interestingly, disappointment in automated translation in the 1960s was not an isolated event. Similar “AI winters” occurred with respect to connectionism (1970s), expert systems (1990s), and other AI topics.

So, although there is tremendous interest in HIT, and even good evidence that it can be useful, some will certainly be disappointed with the results. A recent report by the National Research Council concluded that “. . . current efforts aimed at the nationwide deployment of health care IT will not be sufficient to achieve the vision of 21st century health care, and may even set back the cause if these efforts continue wholly without change from their present course” (NRC, 2009). Thus, there is also good reason for concern that HIT (and the field of biomedical informatics, in general) may be headed for a bust. However, an “HIT winter” would be unfortunate, because there are real benefits to pursuing research and implementation of HIT.

The Semantic Gap

Loosely speaking, philosophers who study information draw a distinction between data (syntax) and information, defined as meaningful data (i.e., data + meaning or, alternatively syntax + semantics) (Floridi, 2005). The fundamental problem is that existing technology can store, manipulate, and transmit data but not information. Thus the utility of HIT is limited to the extent to which data approximates meaning, and, unfortunately, there is a large gap between health care data and health care information. Because the difference between data and information is meaning (semantics), we call this the “semantic gap.”

Interestingly, Claude Shannon hinted at this issue in 1948 in a seminal paper, “A Mathematical Theory of Communication.” This “mathematical theory of communication” came to be known as information theory. Shannon wrote that “[f]requently, the messages have meaning; that is they refer to or are correlated according to some system with certain physical or conceptual entities. These semantic aspects of communication are irrelevant to the engineering problem.” Thus, Shannon’s “information theory” explicitly refers to data rather than information in the philosophical sense.

Consider the differ-ences between banking data and health care data, an account at a bank versus a patient’s record (Table 1). One difference is that concepts relevant to health are vague compared to banking concepts. The proper interpretation of the symbols relevant to health care requires significant background knowledge. For example, a patient can be “sick” in many ways, including derangements in vital signs (e.g., extremely high or low blood pressure), prognosis associated with a diagnosis (e.g., any patient with myocardial infarction [heart attack] is sick), and other factors. Two clinicians who are asked to describe the same “sick” individual may legitimately focus on different facts or data.

Table 1

In contrast, the balance in a bank account (e.g., $1,058.93) is relatively objective and is captured by the symbols. If we assume that all transactions (credits and debits) to the account are in the same units (dollars and not pounds or Euros), we need only the numbers and the mathematical operations of addition and subtraction to compute and report the balance. Even though these symbols abstract away the rich semantic complexity of the balance, such as its current purchasing power or that the money can be used to purchase goods and services, this is of no consequence to the successful automation of bank accounts. Thus data-manipulating machines (IT) are much better suited to manipulating bank accounts than they are to manipulating clinical descriptors.

Twenty-five years ago, S. Marsden Blois (1984) argued that the difficulty of using computers in medicine was due to the nature of medical concepts and medical descriptions. Most medical concepts do not have explicit definitions in terms of necessary and sufficient conditions. Thus they are difficult to describe in the formal languages required by computer systems. For instance, a 2000 definition of a myocardial infarction (heart attack) is nine pages long and contains many imprecise terms, such as: “prolonged,” “usually,” and “experienced observer.” Other medical concepts, such as “sharp pain,” may be even more difficult to map to formal representations.

Social and Administrative Barriers to the Adoption of Health Information Technology

Manipulating data instead of information has many consequences for HIT. The problem for American clinics and hospitals is not usually a shortage of computers. Most hospitals and even small private practices use computers to manage financial and administrative data, and many hospitals have functioning e-mail systems and maintain a Web presence. In addition, many clinicians use personal digital assistants (McLeod et al., 2003), and some communicate with patients via e-mail. In contrast, however, most clinical records are kept on paper.

There are many barriers to the adoption of HIT. Hospitals that have not implemented electronic medical records most frequently cite financial concerns, including the lack of adequate capital for purchasing equipment (74 percent), maintenance costs (44 percent), and unclear return on investment (32 percent) (Jha et al., 2009). Additional barriers include a mismatch between costs and benefits, cultural resistance to change, lack of an appropriately trained workforce to implement HIT, and many others (Hersh, 2004).

To some, clinicians’ resistance to computerization appears to be irrational. However, given the mixed evidence regarding the benefits of HIT, caution seems increasingly reasonable. Thus many clinical enterprises are not computerized because of rational skepticism about the costs and benefits of current HIT, not because of an irrational resistance to technological progress.

Research Challenges

Significant research will be necessary to address serious problems before HIT becomes more attractive to clinicians. Many of these problems are outlined in a recent National Research Council report (NRC, 2009). First, there is a mismatch between what HIT can represent (data) and concepts relevant to health care (data + meaning). This very difficult, fundamental challenge subsumes multiple AI problems (e.g., context or common sense) that have proven very difficult to solve. HIT can manipulate form, but not meaning—hence the term “formal methods.” Until we have true information technology, rather than data technology, the benefits of HIT will be limited to applications in which formal methods (i.e., methods that manipulate form) suffice.

A second research challenge is to define appropriate applications for HIT, as well as policies, procedures, and methods of implementation. Clearly, HIT can be helpful in many ways. For example, computerized alerts and reminders can improve compliance with preventive-care guidelines (Shea et al., 1996) and may be cost effective when used in this way (Bernstam et al., 2000). Similarly, examples of reductions in the number of medication errors and other benefits have been published (e.g., Bates et al., 2001). Therefore, in spite of its limitations, current HIT can be useful when applied to suitable problems.

 A third research challenge is to evaluate HIT as a clinical intervention. An instructive example is that a commercial electronic health record was associated with increased mortality at one institution (Han et al., 2005), but no such association was found at another institution that implemented the same system in a similar care setting (Del Beccaro et al., 2006). Thus outcomes depend on the interplay among HIT, its implementation at a particular institution, and the nature of the institution (e.g., workflow, patient mix, policies, availability of specialists, etc.).

A computer system cannot be considered in isolation. Its effects must be evaluated in the context of a specific organization. Any system that can affect clinical decisions has the potential to worsen as well as to improve outcomes. Therefore, these systems should be evaluated as clinical interventions, just as drugs, medical devices, and procedures are evaluated.

Fourth, HIT must augment human cognition and abilities. This has been elegantly expressed as the “fundamental theorem of informatics”: human + computer > human (Friedman, 2009). In other words, there must be a clear and demonstrable benefit from HIT. Clearly, it can be beneficial in some situations, and in some ways human cognition and computer technology are complementary. Computers excel at precise, efficient manipulation of data, whereas we excel at discovering, storing, and processing meaning. Thus there are tremendous opportunities for effective human-machine collaboration. For example, monitoring (e.g., waveforms) is much easier for computers than for humans. In contrast, reasoning by analogy across domains is natural for humans but difficult for computers.

Defining scenarios, with all relevant parameters, in which HIT is beneficial and demonstrating that using HIT is reliably beneficial in these scenarios remains a research challenge. In its present form, HIT will not transform health care the same way that IT has transformed other industries, partly because of the large semantic gap between health data and health information (concepts). In addition, it is worth noting that many problems with health care will only be solved by changes in health care policy, financing, and so forth.

To address the research challenges described above will require unprecedented collaboration among disciplines that have traditionally worked independently and have fundamentally different methods, values, and domains of study. Nevertheless, many promising interdisciplinary approaches have been developed. For example, seemingly simple safety devices, such as checklists, which were pioneered in aviation, have been applied to health care with dramatic results (Pronovost et al., 2006). Statistical process control, simulation, and other engineering methods have also been successfully applied to certain aspects of health care (NAE and IOM, 2005).

Health Information Technology and U.S. Competitiveness

 True HIT (i.e., health information technology, not health data technology) is critical for U.S. competitiveness in biomedicine—both for biomedical research and for clinical care. Clinical trials are increasingly being conducted in countries with large populations (i.e., large subject pools) and lower regulatory barriers (compared to those in the United States), such as India (Glickman et al., 2009). Barring substantial changes in our values, privacy concerns, and expectations, we simply cannot compete.  For example, because of privacy concerns, the United States has no universal patient identifier. As a result, it is very difficult to identify subjects across clinical trials or patients who move between hospitals and other care settings. In contrast, unique patient identifiers in other countries have greatly facilitated clinical research.

Similarly, the high cost of health care in the United States encourages “medical tourism.” Many Americans travel abroad for care that is too expensive for them to obtain in the United States (Wapner, 2008). Some foreign hospitals actually specialize in providing care to Americans who come for a high-cost procedure, such as coronary artery bypass surgery.

True HIT can help address both of these problems. If we can collect clinical information (meaningful data) as a byproduct of routine care, we can then learn from experience, rather than relying solely on clinical trials. In parallel, we can leverage this information to improve care processes. Thus we would fulfill the promise of HIT described by President Obama.

Conclusions

Clearly we must improve health care in fundamental ways, and HIT will be important in transforming the health care system. However, disappointment seems inevitable, because the promises made on behalf of HIT are not likely to be fully realized in the near future. Historical precedents for such cycles of enthusiasm and disappointment with technology include AI, for which boom and bust cycles appear to be the rule rather than the exception.

Realizing the promise of HIT to improve health care will require an unprecedented level of collaboration among communities that have traditionally had little in common, speak different languages, and have very different world views. Thus we are faced with both challenges and opportunities to find fresh perspectives on fundamental problems in the health care domain. In the process, we may also solve some fundamental information (i.e., computer science) problems related to context and meaning.

References

Bates, D.W., M. Cohen, L.L. Leape, J.M. Overhage, M.M. Shabot, and T. Sheridan. 2001. Reducing the frequency of errors in medicine using information technology. Journal of the American Medical Informatics Association 8(4): 299–308.

Bernstam, E.V., H.R. Strasberg, and D.L. Rubin. 2000. Cost-Benefit Analysis of Computer-based Patient Records with Regard to their Use in Colon-Cancer Screening. Presented at the Asia Pacific Medical Informatics Conference, Hong Kong, China. Available online at http://bmir.stanford.edu/file_asset/index.php/145/BMIR- 2000-0847.pdf.

Blois, M.S. 1984. Information and Medicine: The Nature of Medical Descriptions. Berkeley, Calif.: University of California Press.

Chaudhry, B., J. Wang, S. Wu, M. Maglione, W. Mojica, E. Roth, S.C. Morton, and P.G. Shekelle. 2006. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Annals of Internal Medicine 144(10): 742–752.

Del Beccaro, M.A., H.E. Jeffries, M.A. Eisenberg, and E.D. Harry. 2006. Computerized provider order entry implementation: no association with increased mortality rates in an intensive care unit. Pediatrics 118(1): 290–295.

Floridi, L. 2005. Semantic conceptions of information. In Stanford Encyclopedia of Philosophy. Available online at http://plato.stanford.edu/entries/information-semantic/ .

Friedman, C.P. 2009. A “fundamental theorem” of biomedical informatics. Journal of the American Medical Informatics Association 16(2): 169–170.

Glickman, S.W., J.G. McHutchison, E.D. Peterson, C.B. Cairns, R.A. Harrington, R.M. Califf, and K.A. Schulman. 2009. Ethical and scientific implications of the globalization of clinical research. New England Journal of Medicine 360(8): 816–823.

Han, Y.Y., J.A. Carcillo, S.T. Venkataraman, R.S. Clark, R.S. Watson, T.C. Nguyen, H. Bayir, and R.A. Orr. 2005. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics 116(6): 1506–1512.

Hersh, W. 2004. Health care information technology: progress and barriers. Journal of the American Medical Association 292(18): 2273–2274.

Hutchins, J. 2006. Machine Translation: History. Pp. 375–383 in Encyclopedia of Language and Linguistics, second edition, edited by K. Brown. Burlington, Mass.: Elsevier.

Jha, A.K., C.M. DesRoches, E.G. Campbell, K. Donelan, S.R. Rao, T.G. Ferris, A. Shields, S. Rosenbaum, and D. Blumenthal. 2009. Use of electronic health records in U.S. hospitals. New England Journal of Medicine 360(16): 1628–1638.

 Joint Commission. 2008. Safely Implementing Health Information and Converging Technologies. Available online at http://www.jointcommission.org/SentinelEvents/ SentinelEventA lert/sea_42.htm.

Koppel, R., J.P. Metlay, A. Cohen, B. Abaluck, A.R. Localio, S.E. Kimmel, and B.L. Strom. 2005. Role of computerized physician order entry systems in facilitating medication errors. JAMA 293(10): 1197–1203.

Levenson, N.G., and C.S. Turner. 1993. An investigation of the Therac-25 accidents. IEEE Computer 26(7): 18–41.

Littlejohns, P., J.C. Wyatt, and L. Garvican. 2003. Evaluating computerised health information systems: hard lessons still to be learnt. BMJ 326(7394): 860–863.

McLeod, T.G., J.O. Ebbert, and J.F. Lymp. 2003. Survey assessment of personal digital assistant use among trainees and attending physicians. Journal of the American Medical Informatics Association 10(6): 605–607.

NAE and IOM (National Academy of Engineering and Institute of Medicine). 2005. Building a Better Delivery System: A New Engineering/Health Care Partnership, edited by P.P. Reid, W.D. Compton, J.H. Grossman, and G. Fanjiang.. Washington, D.C.: National Academies Press.

NRC (National Research Council). 1966. ALPAC, Language and Machines: Computers in Translation and Linguistics. Washington, D.C.: National Academy Press.

NRC. 2009. Computational Technology for Effective Health Care: Immediate Steps and Strategic Directions, edited by W.W. Stead and H.S. Lin. Washington, D.C.: National Academies Press.

Pronovost, P., D. Needham, S. Berenholtz, D. Sinopoli, H. Chu, S. Cosgrove, B. Sexton, R. Hyzy, R. Welsh, G. Roth, J. Bander, J. Kepros, and C. Goeschel. 2006. An intervention to decrease catheter-related bloodstream infections in the ICU. New England Journal of Medicine 355(26): 2725–2732.

Shannon, C.E. 1948. A mathematical theory of communication. Bell System Technical Journal 27(2): 379–423, 623–656.

Shea, S., W. DuMouchel, and L. Bahamonde. 1996. A meta-analysis of 16 randomized controlled trials to evaluate computer-based clinical reminder systems for preventive care in the ambulatory setting. Journal of the American Medical Informatics Association 3(6): 399–409.

Wapner, J. 2008. American Medical Association provides guidance on medical tourism. BMJ 337: a575.

Weiner, J.P., T. Kfuri, K. Chan, and J.B. Fowles. 2007. “e-Iatrogenesis”: the most critical unintended consequence of CPOE and other HIT. Journal of the American Medical Informatics Association 14(3): 387–388; discussion 389.
 

About the Author:Elmer V. Bernstam is an associate professor in the School of Health Information Sciences and in the Department of Internal Medicine, University of Texas Health Science Center at Houston. Todd R. Johnson is associate professor, School of Health Information Sciences, University of Texas Health Science Center at Houston.