The Changing Face of Engineering Education

ABET EC2000 has radically changed the evaluation of undergraduate engineering programs.

Most engineers today are likely to recall their undergraduate years as a period of intense mathematical and theoretical study. Until the 1940s, however, the engineering curriculum at most colleges and universities was mostly practical, with the emphasis on engineering design rather than on engineering science and mathematical applications. The field, and undergraduate education programs, changed dramatically with increased federal support for university research after World War II. As engineering schools hired faculty to teach and conduct scientific research, the number of faculty members with industry experience declined. Eventually, courses in advanced mathematics and theory replaced practical courses in machining, surveying, and drawing (Prados et al., 2005).

The strong emphasis on technical knowledge and skills, and resulting improvements in defense-related technologies and applications, served the United States well during the war and postwar years, but, by the 1980s, economic shifts from defense to commercial applications left engineering employers dissatisfied. New graduates were technically well prepared but lacked the professional skills for success in a competitive, innovative, global marketplace. Employers complained that new hires had poor communication and teamwork skills and did not appreciate the social and nontechnical influences on engineering solutions and quality processes (McMasters, 2004; Todd et al., 1993). Several national reports recommending changes in engineering education appeared (e.g., ASEE 1987; NRC, 1985; NSB, 1986; NSF, 1989).
_______________________________

EC2000 emphasizes learning
outcomes and accountability.
_______________________________

Despite a significant investment by the National Science Foundation (NSF) in engineering education, engineering schools were slow to respond to these criticisms, and deans of many colleges of engineering argued that rigid accreditation standards focused on curricular requirements and credit hours were hindering their efforts to make necessary changes. In 1992, a representative of two groups of deans of major engineering schools met with the leadership of ABET, then the Accreditation Board for Engineering and Technology, to express their concerns. Some deans suggested that a different accreditation organization was needed.

Inside ABET, members of the agency’s Industry Advisory Council echoed the criticisms of members of the engineering community, and ABET’s leadership responded. In 1992, the outgoing and incoming presidents of ABET jointly established an Accreditation Process Review Committee (APRC) to advise the organization on making accreditation criteria more flexible without compromising educational quality. Members of APRC were drawn from ABET and its Engineering Accreditation Commission, as well as from academia and industry. APRC identified three major barriers to change: criteria, process, and participation.

In 1994, with funding from NSF and industries represented on the ABET Industry Advisory Council, ABET sponsored three consensus-building workshops to address these concerns. More than 125 individuals, including university presidents, deans, and faculty members; industry leaders; private practitioners; professional and technical society liaisons and executive directors; state registration board members; and government researchers and regulators in technical fields participated in the workshops with ABET leaders, commissioners, and board members. Recommendations from the workshops, published in A Vision for Change in early 1995, became catalysts for the development of new accreditation criteria, which were circulated to the engineering community a few months later. Following a period of public comment, the ABET Board of Directors approved Engineering Criteria 2000 (EC2000) in 1996.

The new criteria radically altered the evaluation of undergraduate engineering programs, shifting the emphasis from curricular specifications to student learning outcomes and accountability. Under EC2000, engineering programs must define program objectives to meet their constituents’ needs. Rather than taking a cookie-cutter approach, the new criteria accommodate differences and innovations in programs. To ensure accountability, each engineering program is required to implement a structured, documented system for continuous improvement that actively and formally engages all of its constituents in the development, assessment, and improvement of academic offerings. Programs must publish specific goals for student learning and measure their achievement to demonstrate how well these objectives are being met. ABET was one of the first accrediting bodies to implement such a radical change in its accreditation philosophy.

Engineering Education Then and Now
To evaluate the impact of the new outcomes-based criteria on engineering graduates, ABET commissioned the Center for the Study of Higher Education at Pennsylvania State University to conduct an assessment. The study addressed two issues: (1) the impact (if any) of EC2000 on engineering students’ preparation for entering their professions and (2) whether engineering programs had changed their organizational and educational policies and practices in ways consistent with the EC2000 specifications (Lattuca et al., 2006).

Following the conceptual model developed for the study (Figure 1 - see PDF version for all figures), if EC2000 had the desired effects, one would expect to see a variety of changes rippling through the engineering educational process. The most immediate changes would be in engineering programs, as faculty members revised their curricula and instructional practices to promote the learning outcomes specified in EC2000 Criterion 3.a–k (Figure 2). One might also expect to see a shift in faculty culture toward greater involvement in learner-centered activities, such as outcomes assessment and professional development to improve teaching. Similarly, one might anticipate changes in administrative policies and practices, such as more emphasis on teaching and learning in hiring, promotion, and tenure decisions.

If all of the program changes were under way, the logic of the conceptual model suggested that improvements would be apparent in students’ engineering education-related experiences. Like the changes in programs, the improvements would be consistent with achieving the learning outcomes specified by EC2000. Finally, such a transformation in engineering education over the past decade would be reflected in higher knowledge and skill levels among recent graduates, compared to those of their predecessors, on measures of the EC2000 outcomes.

A Pr?cis of the Study Design and Methods
The study design for Engineering Change (Lattuca et al., 2006) was based on information from graduates (both pre- and post-EC2000), faculty members, program chairs, deans, and employers; the goal was to obtain a 360-degree view of changes in engineering education since the implementation of EC2000. A two-stage, disproportionate, stratified, random-sampling design ensured that sample sizes were adequate and that program disciplines, accreditation review schedules, and participation in an NSF-funded coalition during the 1990s were all taken into account. Adjustments were made to include institutions that serve historically underrepresented populations.

Data were collected from 40 colleges or schools of engineering that offer more than 200 engineering programs in aerospace, chemical, civil, computer, electrical, industrial, and mechanical engineering. Analyses were based on survey information from 1,243 faculty members (42 percent response rate), 147 program chairs (72 percent), 5,494 graduates in the class of 1994 (42 percent), 4,330 graduates of the class of 2004 (34 percent), 39 deans (98 percent), and 1,622 employers (population size and, thus, response rate unknown). By social science standards, these response rates were highly respectable, and the numbers of respondents were more than adequate for valid analyses. Statistical adjustments were made to correct for response bias by sex, NSF coalition participation, and discipline among faculty members, as well as for sex and discipline among graduates. Adjustments were also made for varying response rates among institutions. For a detailed description of the study design and methods, see Lattuca et al. (2006). The major findings are summarized below.

Changes in Programs
Curriculum and Instruction
According to program chairs and faculty members, engineering program curricula have changed considerably “over the past seven to 10 years.” Although few programs have relaxed their emphasis on foundational skills in mathematics, science, and engineering science, both program chairs and faculty members reported increased emphasis on nearly all of the professional skills and knowledge sets associated with EC2000 Criterion 3.a–k. For example, 75 percent or more of program chairs reported “some” or “significant” increases in emphasis on communication, teamwork, use of modern engineering tools, technical writing, lifelong learning, and engineering design. Similarly, more than half of the faculty respondents reported “some” or “significant” increases in their emphasis on the use of modern engineering tools, teamwork, and engineering design in courses they teach regularly.

Teaching methods also changed substantially. One-half to two-thirds of faculty respondents said they increased “some” or “significantly” their use of active learning approaches, such as group work, design projects, case studies, and application exercises. They also relied less on lecturing and textbook problems (Figure 3).

Faculty Culture
Faculty members reported increased involvement in the past 10 years in assessing student learning and using assessment results to guide program improvement. Program chairs, moreover, reported high levels of faculty support for assessments and related decision-support practices. More than 75 percent of program chairs estimated that either “more than half” or “almost all” of their faculty members supported efforts to ensure continuous quality improvement, and more than 60 percent of program chairs reported “moderate” to “strong” support among their colleagues for assessing student learning. Faculty members corroborated that finding. Nearly 90 percent reported being personally involved in outcomes assessment, and more than half reported “moderate” to “a great deal” of personal effort in this area. Nearly 70 percent thought their level of effort was “about right.”

Another change in faculty culture included more engagement in professional development activities focused on teaching and learning. More than two-thirds of faculty respondents reported that, compared to seven to 10 years ago, they were reading more about teaching, and about half reported more frequent involvement in professional development activities, such as workshops on teaching, learning, and assessment and projects to improve engineering education. Depending on the activity, one-fifth to one-quarter of faculty members said they had increased their teaching-and-learning-related professional development efforts in the past five years.

Administrative Practices and Policies
The institutional reward system may be the single most important influence on how much time and energy faculty members devote to their teaching, research, and service. Thus, it is important to know if faculty members believe their efforts to improve teaching and undergraduate education are important considerations in decisions about promotion, tenure, and merit-based salary increases.

About half of the program chairs and faculty surveyed said they saw no change in the reward system over the past decade. Roughly 25 percent of faculty respondents said there was less emphasis on teaching since the mid-1990s; another 25 percent, however, said that the emphasis on teaching in the reward system had increased “some” or “significantly” over the same period. Faculty perceptions varied by academic rank. Senior faculty members were more likely to report an increase in emphasis on teaching in promotion and tenure decisions; untenured faculty were more likely to report less emphasis. About one-third of program chairs said the emphasis on teaching in promotion, tenure, and salary and merit decisions had increased over the past decade.

Changes in Student Experiences
According to the logic of the study’s conceptual framework, changes relating to curriculum and instruction, faculty culture, and program practices and policies should be reflected in the experiences of graduates. Students who graduated in 2004 differed significantly from their predecessors in eight of 10 experiences inside and outside the classroom. The 2004 graduates reported more active engagement in their learning, more interaction with instructors, more faculty feedback on their work, more time spent studying abroad, more international travel, more involvement in engineering design competitions, and more openness in their programs to new ideas and people. Although the differences tended to be small, they persisted even after adjustments were made for an array of graduate and institutional characteristics.¹

The 2004 graduates, however, reported experiencing a less welcoming diversity climate than their predecessors. This finding may be attributable to several factors: differences in the sex and/or racial/ethnic mix of the two cohorts, differences in awareness of diversity issues, and/or differences in the willingness to discuss and challenge prejudices or discrimination. The evidence provides no basis for evaluating these possible explanations. The 1994 and 2004 groups gave the same evaluations of their instructors’ teaching skills and the number of hours spent in cooperative or internship activities.

Changes in Learning Outcomes
Given all of these changes, are recent graduates better prepared to enter the profession than their counterparts were a decade earlier? Figures 4, 5, and 6 contrast the achievement levels graduates reported on scales reflecting the Criterion 3.a–k learning outcomes.2 For nine different measures,3 the differences were consistent with the assumption that EC2000 is a force for change in engineering education. All differences were statistically significant (p < .001), ranging from +.07 to +.80 of a standard deviation (sd) (mean effect size = +.36 sd).4 Five of the nine effect sizes exceeded .3 sd, which would be characterized as “moderate” in the social sciences.

The largest differences over the past decade were in five areas: awareness of societal and global issues (effect size = +.80 sd), awareness of ethics and professionalism (+.46 sd), group skills (+.47 sd), and applying engineering skills (+.47 sd). The smallest increases were in the ability to apply mathematics and sciences (+.07 sd). This last difference, although small, is noteworthy because some faculty members and others had expressed concerns that focusing on the development of the professional skills specified in EC2000 might detract from the teaching of the science, math, and engineering science skills that are the foundations of engineering. This finding not only indicates no decline, but shows a slight rise in knowledge and skills in these areas.

Linking Changes and Learning Outcomes
Multivariate statistical analyses provide moderate to strong evidence that changes in programs and student experiences were empirically linked to differences in learning. The evidence for this remained, moreover, even after controlling for a battery of students’ precollege characteristics and for the characteristics of the institutions they attended. Fourteen of 16 program and faculty changes had statistically significant effects on at least one, and as many as five, student experiences, even after controlling for other factors. The most frequent and influential programmatic changes were an increase in emphasis on foundational knowledge and project skills in program curricula, less reliance on traditional pedagogies, an increase in active and collaborative pedagogies, and an increase in programmatic emphasis on assessment for improvement. Although student experiences varied, a consistent pattern emerged. The observed shifts over the past decade in program curricula, practices and policies, and faculty activities and culture related positively, at statistically significant, if sometimes small or moderate, levels, even after other factors were taken into account.

Finally, undergraduate program experiences, both inside and outside the classroom, were clearly linked to what and how much students learned. Nine of 10 measures had statistically significant, positive, and sometimes substantial influences on all nine of the measures of the EC2000 learning outcomes. The clarity of instruction, the amount of interaction with and feedback from instructors, and engagement in active and collaborative pedagogies had the strongest influence on learning.

Out-of-class experiences, however, also influenced student learning in important ways. The most important non-classroom activities were internships and cooperative education experiences, participation in design competitions, and active participation in student chapters of professional societies or associations. These non-classroom activities significantly and positively affected learning in six or more of the nine skill areas measured, although the magnitude of the effects were smaller than those of in-class experiences.

Employers’ Views of Change
The 1,622 employer respondents in the study represented a wide range of geographic locations, industry types, company size, educational attainment, and engineer-evaluation experience. The employer survey asked three primary questions.

1. How important are the 11 Criterion 3 competencies for today’s new hires?
2. How well prepared are today’s graduates (on each of five dimensions reflecting the 11 a–k criteria)?
3. What changes have occurred over the past seven to 10 years in recent graduates’ abilities?

The employer findings indicated that the EC2000 learning outcomes introduced in 1996 continue to be important to employers hiring recent graduates. Seven of 10 employers rated all 11 of the a–k criteria as at least “moderately” important, and at least six of 10 employers rated nine items as “highly important” or “essential.” Moreover, employer support for the outcomes was fairly consistent across engineering fields, geographic locations, company sizes, types of business, and employer job title and educational attainment.

As shown in Figure 7, more than 90 percent of employers thought new engineering graduates were adequately or well prepared to use math, science, and technical skills, and about eight of 10 gave recent graduates passing marks on their ability to solve problems and to learn, grow, and adapt. Three of four employers assessed graduates’ teamwork and communication skills as at least adequate. Moreover, these employers reported modest improvements in the past decade in teamwork and communication skills, as well as in the ability to learn and adapt to changing technologies and society. Employers perceived no change in technical skills in math and science, but some noted a modest decline in problem-solving skills, although eight out of 10 still rated problem-solving skills as at least adequate. Barely half of employers, however, found the understanding of organizational, cultural, and environmental contexts and constraints to be adequate. Moreover, skills in this area, according to employers, appeared to have declined somewhat over the past decade.

Employer assessments were strikingly similar in all engineering fields, industry sectors, and geographic locations. Analyses indicate, however, that employers from larger companies that recruit nationally and hire the most engineers had more favorable assessments of preparation and changes over the past decade than employers from smaller companies that recruit locally and hire fewer engineers. This finding suggests that large national companies are seeing changes first and that some changes are just beginning to be visible to employers.

Conclusions
In the 1980s, employers expressed dissatisfaction with engineering graduates’ professional skills. By the mid-1990s, ABET had implemented a new accreditation philosophy based on assessments of student learning and continuous improvement principles. Today, according to the accumulated evidence in Engineering Change, engineering education in the United States has changed dramatically. Engineering programs and faculty members have reengineered their curricula, teaching methods, professional development practices, program assessment and decision making, and, to some extent, their hiring, promotion, and tenure criteria.

Perhaps most important, graduates in 2004 were measurably better prepared than their counterparts of a decade ago in all of the nine learning areas assessed. The greatest increases were in understanding of societal and global issues, the ability to apply engineering skills, teamwork, and the appreciation of ethics and professional issues—all attributes U.S. engineers need to compete successfully in a competitive, global economy.

Notes

1. Analyses controlled for differences in sex, race/ethnicity, citizenship, native/transfer entry status, full-/part-time enrollment, parents’ education and income, and secondary school academic achievement, as well as for institutional type, size, wealth, Carnegie Classification, EC2000 review schedule, and participation in an NSF-funded coalition.
2. Assessments are based on self-reported skill levels at the time of graduation. Although some individuals have questioned the validity of self-reports, a growing body of research in the past 30 years suggests that self-reported measures of learning and skill development are adequate proxies for objective measures of the same skills. Strauss and Terenzini (2005) provide a detailed description of the development and psychometric characteristics of the study’s criterion measures and a discussion of the validity of self-reports.
3. In factor analyses, two of the 11 scales developed a priori to operationalize the a–k criteria collapsed into other scales, leaving a total of nine scales.
4. An effect size reflects the magnitude of the difference between two means after adjusting for differences in the variability of scores. The effect sizes, reported here in standard deviation (sd) units, can also be expressed as percentile-point differences. Assuming the mean skill level for 1994 graduates on any outcome marks the 50th percentile, an average increase among 2004 graduates of .2 sd is the equivalent of 8 percentile-points higher, or the 58th percentile. Other sample conversions: .4 sd=a 17 percentile-point difference; .6=23 percentile points; .8=29 percentile points, and 1.0=34 percentile points.

References
ABET (Accreditation Board for Engineering and Technology). 1995. A Vision for Change: A Summary Report of the ABET/NSF Industry Workshops. Baltimore, Md.: ABET.
ABET. 1997. Engineering Criteria 2000. Baltimore, Md.: ABET. Available online at: www.abet.org.
ASEE (American Society for Engineering Education). 1987. A National Action Agenda for Engineering Education. Report of an ASEE Task Force. Washington, D.C.: ASEE.
Lattuca, L.R., P.T. Terenzini, and J.F. Volkwein. 2006. Engineering Change: A Study of the Impact of EC2000. Baltimore, Md.: ABET Inc. Executive Summary available online at: www.abet.org/papers.shtml.
McMasters, J.H. 2004. Influencing engineering education: one (aerospace) industry perspective. International Journal of Engineering Education 20(3): 353–371.
NRC (National Research Council). 1985. Engineering Education and Practice in the United States: Foundations of Our Techno-Economic Future. Washington, D.C.: National Academy Press.
NSB (National Science Board). 1986. Undergraduate Science, Mathematics, and Engineering Education: Role for the National Science Foundation and Recommendations for Action by Other Sectors to Strengthen Collegiate Education and Pursue Excellence in the Next Generation of U.S. Leadership in Science and Technology. NSB 86-100. Washington, D.C.: National Science Board.
NSF (National Science Foundation). 1989. Imperatives in Undergraduate Engineering Education: Issues and Actions. Report of an NSF Ad Hoc Task Force. August. Washington, D.C.: National Science Foundation.
Prados, J.W., G.D. Peterson, and L.R. Lattuca. 2005. Quality assurance of engineering education through accreditation: the impact of Engineering Criteria 2000 and its global influence. Journal of Engineering Education 94(1): 165–184.
Strauss, L.C., and P.T. Terenzini. 2005. Assessing student performance on EC2000 Criterion 3. The Changing Landscape of Engineering and Technology Education in a Global World. 2005. ASEE Annual Conference and Exposition, June 12–15, 2005, Portland, Oregon. Available online at: http://www.asee.org/acPapers/2005-1474_Final.pdf.
Todd, R.H., C.C. Sorenson, and S.P. Magleby. 1993. Designing a capstone senior course to satisfy industrial customers. Journal of Engineering Education 82(2): 92–100.