Getting the Most Out of Information Technology

Information technology research must address problems of large-scale systems and social applications if society is to get the most out of the technology.

Continued advances in information technology (IT) have resulted in dramatic transformations in many important societal functions, including commerce, education, health care, and delivery of government services. Networking in particular has enabled the development of increasingly powerful applications that span multiple organizations, linking people and information in unexpected ways. Nevertheless, IT systems have failed to live up to their full potential in many applications-crashing unexpectedly, suffering security breaches, producing unanticipated results, and failing to scale up to meet growing demand or critical requirements. These problems underscore the shortfall in our understanding of many issues related to large-scale IT systems and their applications. They suggest that achieving the promise of IT will require new research efforts. Drawing upon a recent report from the National Research Council (NRC) Computer Science and Telecommunications Board on the future of IT research (2000), this article argues for an expansion of the IT research agenda to address these increasingly important problems and calls for a broader set of constituencies to fund, participate in, and perform IT research to build the base of understanding needed to get the most out of IT.

The movement of IT out of the laboratory and the back rooms of large organizations over the past few decades has allowed computing and communications systems to touch many aspects of personal and work life. Due in large part to the Internet, people routinely use IT to communicate with friends and relatives, plan trips, shop, and enjoy entertainment. Businesses use IT to buy and sell goods and services, keep track of payroll and inventory, facilitate collaborative work, and control production lines. Government agencies use IT to provide benefits and information to citizens, process taxes and grant applications, and procure supplies. A growing number of critical infrastructures, from the nation’s power grid and air traffic control systems to its financial and health care systems, increasingly benefit from (and have become more reliant on) IT. Many consumer products, such as automobiles and refrigerators, integrally incorporate IT to control their operation.

The economic impact of these IT-based transformations is significant. The United States is preeminent in both the supply and the effective use of IT, and many believe that because of IT the United States is experiencing higher economic growth and productivity gains that can be sustained well into the future. The U.S. Department of Commerce estimates that one-third of the nation’s economic growth since 1992 is attributable to firms that offer IT-related goods and services, and that half or more of the nation’s productivity growth in the last half of the decade resulted from the production and use of IT. (2) Some 7.4 million Americans worked in IT-producing industries or in IT-related positions in other industries in 1998, at wages significantly above the national average. All signs point toward a growing need for skilled IT workers, perhaps exceeding supply. (3)

In spite of these remarkable achievements, IT systems continue to garner a number of vexing problems. Large companies and federal agencies experience chronic difficulties in developing, deploying, and operating new IT systems. Recent surveys indicate that only one-quarter of all large-scale system development efforts are completed on time and within budget, and almost 30 percent are abandoned because requirements cannot be met (Johnson, 1999; Standish Group, 1995). Once deployed, many systems lack adequate reliability and security measures to prevent the kinds of problems that afflicted several online stock trading sites and Washington, D.C.’s Metrorail system in the past year (Layton, 1999; Meehan, 2000). Reliability and security are becoming more challenging as the Internet allows the deployment of more tightly interlinked and interdependent IT systems.

Beyond these obvious failures are missed opportunities resulting from either the failure to apply IT where it might benefit or the failure to apply IT most effectively. Many IT systems are not integrated well into the activities they support, undermining attempts to boost productivity. Other applications are inflexible, failing to accommodate changing needs or add new capabilities. IT often stifles or complicates changes in organizations, processes, and products, rather than enabling and simplifying them, as should be the case. Increasing intellectual resources are applied to circumventing or bypassing existing, outdated systems, obviating fresh opportunities. The most demanding applications sometimes have difficulty scaling to rapidly growing user bases.

Large-scale Systems

The causes of these problems are many and varied. Certainly, poor project management or inadequate worker skills and training contribute to many failed development efforts or operational failures. More significant is a lack of fundamental understanding underpinning some aspects of IT and its application. The design of large-scale systems-IT systems with large numbers of heterogeneous elements that interact in complex ways-is still as much art as science, and we have inadequate tools for predicting or analyzing performance. We have no set of design principles that ensure robust, scalable, and flexible large-scale systems. While designers have time-tested principles such as abstraction, layering, and modularity that make complicated systems easier to understand, more powerful approaches are needed as IT systems grow ever larger in scale, more complex, and more dynamic.

While these problems are not new-systems issues have long plagued IT as well as other engineering products-they take on added significance as the scale of IT systems increases and the scope of applications broadens. Future applications will be even larger and more complex, if only because they will be more highly distributed and more reliant on networking. The fact that such problems are easily recognizable today-and responsible for some of the development failures mentioned above-suggests that IT developments will be increasingly constrained without substantial and sustained improvements in fundamental understanding and tools.

Social Applications

Another complication is the increasingly tight coupling between IT systems and the processes they support. IT applications supporting organizational missions and business processes-in industry, government, education, health care, and other areas-are increasingly specialized to fit specific organizational missions. For example, an application supporting concurrent automobile design, with work distributed across several continents, will have some generic elements, such as information access, but also many functions specific to automobiles and the way they are designed. As specialized applications become more tightly intertwined with organizational missions and processes, IT becomes embedded in a human organization (much as an engine controller is embedded in an automobile), raising a number of questions about what work is delegated to people, what is delegated to machines, how the people are organized, how the machines are organized, and how these human and technical organizations relate to one another. A similar phenomenon occurred in the industrial revolution with the organization of manufacturing processes, but such issues now expand to all arenas of human activity. Dare we say we are experiencing the information revolution, and not just the information age?

In social applications of IT, such as concurrent design-applications that involve integral collections of people, organizational structures, and IT systems-the maximum value of IT (by productivity and effectiveness measures) cannot be tapped by simply automating existing processes. Rather, it is important to reconsider all aspects of organizational functions and processes in light of the capabilities of IT. It also becomes appropriate to reexamine IT itself, to see how it can become a more effective resource. From the perspective of research and design, social applications target sociotechnical systems; that is, systems composed of human organizations and networked computing. The IT part is itself a large and complex system, providing additional stimulus for the research into large-scale systems mentioned earlier.

The research community has historically devoted less attention to the challenges of large-scale systems and social applications than to the underlying core technologies. While the nation’s research enterprise continues to thrive-IT companies allocated more than $50 billion to research and development in 1998, and government contributed about $2 billion in research funding-much of this research has focused on the development of new and improved software infrastructure and component technologies (e.g., microprocessors, memory, and communications devices) that underlie modern IT. (4) This research has produced important improvements in core processing, storage, and communications technologies, as well as in the understanding of massive computations and the management of large data sets. As a result, we have the building blocks with which to develop a broad array of systems. Nevertheless, the IT research base needs to expand more fully into systems and social applications if continued improvements in these building blocks are to be exploited effectively.

Social applications are an important target for research. Although they are increasingly pervasive-even universal-they are poorly understood. There are many opportunities for innovative new ways to apply IT more effectively, to design work processes and organizational structures, and to structure technology and tools to more effectively support applications and mitigate design challenges. Such research could also point the way to new opportunities for IT infrastructure that aids and supports applications.

Unfortunately, these opportunities for applications research are underappreciated in academic, industrial, and government research organizations. Too often, IT is used to automate rather than transform processes, or it is assumed that IT is a fixed point that can be incorporated into a social application but not changed in any way. Typical of this is the current rage in incorporating the Web into business functions without reflecting on whether the Web is in fact the best foundation.

The interactions between IT and applications should flow in two directions. IT should cause a reexamination of application domains with the goal of applying it most effectively, and applications should point to opportunities to improve IT. After all, IT rarely suffers from fundamental physical limitations-it is what we make of it, with few constraints. Most limits or shortcomings are conceptual, the result of restricted understanding. Today’s IT is surely not the best it can be.

Research into social applications can take many forms. For example, it can focus on generic requirements that arise in a number of application domains, such as achieving flexibility to meet changing needs and requirements. Another generic requirement is personal privacy-one of the grand challenges of the information age. How can legitimate functions be realized (with assured user control over the dissemination of information) without a complicated patchwork of laws and regulations? Surely, it is feasible to achieve this through improvements in IT technology, at least to a greater extent than we see today.

Research can also focus on specific application domains. One goal is to positively affect the particular domain through the effective application of IT. Such research can also uncover important relationships among generic issues, for example, how flexibility interacts with privacy. Specific application domains can also be a driver, pointing to the most stringent requirements.

Conducting effective research into social applications requires the participation of a wider base of researchers. In particular, it must involve people and organizations that are major end users of IT, because their knowledge is integral to understanding the functions to be achieved, the application requirements and how they may be modified, and any constraints on organizations and processes. For example, what does flexibility really mean? Does it mean the ability to make two applications work together, or to allow two organizations to merge their systems, or to allow an application to be dramatically reshaped? What does privacy mean in different circumstances, and what forms of end-user control over information dissemination are required? Considering a range of application domains can give insight to these and similar questions. Such research also needs to include technology specialists, because only they fully appreciate the range of options that IT can provide. End users and technologists must collaborate, because the best opportunity is to mold the technical and human/organizational aspects of the sociotechnical systems together.

Encouraging Interdisciplinary Research

Other academic disciplines should participate as well. All the engineering disciplines, as well as other professional disciplines such as business and medicine, represent opportunities for social applications. The social sciences and law have many insights to offer and many opportunities to address challenging issues surrounding social applications. Social application research is not only interdisciplinary; it also requires a prolonged and deep intermingling that transcends the simple application of one discipline to another.

Encouraging the right kinds of interdisciplinary research to make progress on problems of large-scale systems and social applications requires a diversity of organizational structures and approaches. Work is needed to develop the theoretical underpinnings of systems and new architectural approaches to design and software development processes, as well as on the range of interactions between IT and social applications. Some of this work will need to be conducted by individual investigators; other work will require small interdisciplinary research teams or larger research centers that can launch broader, longer-term collaborations. Researchers should be allowed to identify and structure mechanisms best suited to the context and goals.

Unfortunately, there are obstacles to interdisciplinary research. There are cultural differences in the way researchers identify and define problems, in their experimental approaches, in their language and terminology, and in their expectations of outcomes. Administrative structures that reinforce disciplinary boundaries also get in the way. Within universities, in particular, the disciplinary nature of academic departments and research centers results in review and promotion practices that sometimes undervalue work across disciplinary boundaries. These impediments can be especially discouraging to junior faculty concerned about long-term career prospects and tenure. Although these issues are often of less concern in industrial research laboratories, which often organize research around problem areas rather than disciplines, they can nevertheless arise in less apparent ways.

Several steps can be taken to address these concerns. One approach is for universities (or industrial organizations) to establish new research and teaching units specifically targeted at interdisciplinary IT research, as is happening at some forward-looking institutions. Such schools, departments, and research centers serve not only to recognize and legitimize interdisciplinary research, but also to enable the establishment of review and tenure criteria that are aligned with the interdisciplinary area of inquiry. Another approach is to help interdisciplinary research flourish within existing academic departments by developing and promulgating review and evaluation criteria that explicitly recognize and encourage work across disciplinary boundaries without sacrificing quality and rigor.

Additional steps will be needed to help engage end users more effectively in IT research, but numerous issues need to be addressed. Should, for example, end-user organizations set up research organizations to address pertinent social applications? Perhaps so, but they should probably do this in a measured way, since they have little experience in managing such research and may have difficulty in attracting top IT research talent.

More promising for the near term are mechanisms that directly involve end-user organizations in IT research but retain the actual performance of the research in existing industrial, government, and, especially, academic research organizations. End-user representatives can sit on advisory boards, or make personnel available for consultation. They can allow researchers to have access to their facilities to interview workers and managers, observe work in progress, or conduct experiments in a realistic environment.

End-user organizations should also fund research into social applications, whether internally or in academia. Such new sources of funding are needed if the research agenda is to expand into social applications without detracting from continued research in the component technologies that are needed to keep progress in information technology going. Funding research will also allow end users to influence the research agenda and help produce strong returns on investment if the research is performed well. Many end-user organizations-from government mission agencies, such as the Internal Revenue Service and Federal Aviation Administration, to large corporations-make significant expenditures on the development of IT applications. Even modest improvements in their ability to design and deploy large-scale systems and social applications could result in significant development cost savings and more effective applications. Companies that operate large IT systems, including network operators and application service providers, will also see benefits from their investments in IT research, especially in large-scale systems research.

Organizations in particular application domains and vertical industries can also pool their resources to fund precompetitive research. They could also establish new research organizations patterned after some of the more successful existing examples like the Electric Power Research Institute, the Cable Research Laboratories, or the Semiconductor Research Corporation. This approach would allow better coordination of expenditures and create a critical mass of researchers addressing particular problems.

Perhaps the greatest obstacles to more interdisciplinary research and the greater involvement of end users in IT research are cultural. Today, top IT researchers are often unaware of the opportunities for making a major impact in application areas and of the serious and interesting challenges they face. Similarly, end-user organizations are often unaware of the opportunity for research to redress their serious challenges, or of the major opportunities arising from more effective use of IT that can be uncovered with research. Exceptions include the military, space exploration, and medicine, among other application domains, which long ago recognized the applicability of research (although not always related to IT) to their mission, and have realized tremendous benefits from these investments.

A hopeful precedent is scientific computing. There is a long history of successful collaboration between natural and social scientists and the IT-related disciplines (e.g., computer science and engineering, electrical engineering, and information sciences) centered on modeling, simulation, and data extraction and analysis. This collaboration has profoundly affected high-performance computing technology and is having a major impact on the practice of science itself. This collaboration was aided by a greater cultural affinity and mutual appreciation among computer scientists and other scientists.

Other hopeful historical precedents include mathematics, economics, and the social sciences, which have become increasingly prevalent components of many other disciplines. Like mathematics and economics, IT will increasingly become an integral part of many disciplines, with domain specialists acquiring a strong IT background. Conversely, the social sciences can become a more integral part of IT, just as social scientists have complemented engineering disciplines in areas such as urban planning, transportation, architecture, and structural engineering. All this is a natural-and hardly unprecedented-result of the increasing pervasiveness of IT.

Information technology is one of the most exciting and influential developments of our age, and together with other new technologies, it strongly underpins the success of the nation. In light of its increasing influence and impact, evident and growing problems in the application of IT must be taken seriously. The nation needs to invest increased resources for research, particularly in problem areas that are today largely ignored. The areas of large-scale systems and social applications both merit increased attention from researchers in application domains and in information technologies. Increased funding will be needed, especially from (or on behalf of) the end-user organizations and service providers that will be the most direct beneficiaries of research. The returns on investment could be enormous. As almost every citizen and organization depends increasingly on IT, the importance of improved understanding and better outcomes grows.


Acknowledgments

The Directorate for Computer and Information Science and Engineering of the National Science Foundation supported the NRC study summarized here. Other members of the study committee (the Committee on Information Technology Research in a Competitive World) contributed many of the ideas expressed in this article and in the full report, Making IT Better (NRC, 2000). Committee members Robert Sproull (NAE) and Stewart Personick (NAE) supplied valuable comments on this article.


Notes

1. The views expressed in this article reflect the personal perspectives of the authors and are not statements of the National Research Council, the National Academies, or the University of California, although they draw upon the authors’ contributions to the NRC report, Making IT Better.

2. All economic data cited in this paragraph are derived from the U.S. Department of Commerce (2000).

3. The NRC’s Computer Science and Telecommunications Board plans to release a study on IT workforce issues in the fall of 2000. Additional information is available online at http://www4.nationalacademies.org/cpsma/itwpublic2.nsf.

4. Estimates of industry and government spending on research and development related to information technology were made by members of the study committee that produced the report Making IT Better. The industry figures derive from data compiled for the National Science Foundation by the U.S. Bureau of the Census for the following six industries: office, computing, and accounting machines; communications equipment; electronic components; communications services; computing and data processing services; and wholesale trade of professional and commercial equipment and supplies.


References

Johnson, J. 1999. Turning chaos into success. Software Magazine 19(3):30-39. Available: http://www.softwaremag.com/archive/1999dec/success.html [11 August 2000].

Layton, L. 1999. Computer failure puzzles Metro: Opening delayed, rush hour slowed. Washington Post, 25 September, p. B1.

Leunig, E. 1999. Schwab suffers repeated online outages. CNET News.com (22 October). [Online.] Available: http://news.cnet.com/news/0-1007-200-922368.html [11 August 2000].

Meehan, M. 2000. Update: System outages top online brokerage execs’ concerns. Computerworld (4 April). [Online.] Available: http://www.computerworld.com/cwi/story/0,1199,NAV47_ STO43763 ,00.html [11 August 2000].

National Research Council (NRC). 2000. Making IT Better: Expanding Information Technology Research to Meet Society’s Needs. Washington, D.C.: National Academy Press.

Standish Group International. 1995. The Chaos. [Online.] Available: http://standishgroup.com/visitor/chaos.htm [11 August 2000].

U.S. Department of Commerce. Economics and Statistics Administration. 2000. Digital Economy 2000. Washington, D.C.: Department of Commerce.