Download PDF Spring Bridge Issue on Engineering and Climate Change March 15, 2020 Volume 50 Issue 1 The seven articles in this issue cannot cover all engineering-related aspects of climate change, but they highlight several areas of concern. Invisible Bridges: Deep Unlearning Friday, March 13, 2020 Author: Guru Madhavan There’s a clear line between content analysis and cussing. IBM’s Watson crossed that line. While preparing for Jeopardy!, the famous “question-answering” system ingested a smorgasbord of content, including the Urban Dictionary, a sourcebook of slang. Watson was “learning”—and learned to swear.[1] The engineers were nonplussed. What to do about the expletives? Simple: just block, ban, censor. Control the input, control the output. End of story. But is it? A doubleness seems to define much of what we try to do with artificial intelligence: while we want machines to learn as humans do, we can filter their input to privilege one perspective over another. What machines learn is in part a function of what else they could learn. With this capacity come many conveniences. Our devices are companionable. Their status updates keep us in the “now.” Search engines have become “searching” engines, ever active without a command. We drive electrons with our thumbs and voice to summon a ride share or get turn-by-turn directions. Looking for movies to stream? Seamless. Podcasts? Delivered. Designer pizza or Ethiopian stew? Enjoy. Remixed 1970s funk? Here you go (and you might also like 1990s Trip Hop). Set your smart thermostat? Cool. Execute a precision strike on a foe? Bam. Out of this mining and mingling emerges the AI version of being swept by currents of data akin to pollen gliding in the winds. We have empowered—and come to expect—greedy algorithms to conduct our work. In doing so, in important ways we have elbowed humans from the equation. This may be an unstated strategy in engineering, but every technological push forward evokes a social pushback. Langdon Winner termed this “mythinformation”—the hype to the public about artificial intelligence (AI) confronting the public’s distrust of AI. Writing in 1984, Winner observed that AI’s “present course is influenced by…the absent mind” (p. 596). Think about humans’ ability to understand what we are reading, let alone learning. “Dark patterns” online make us interpret one thing when what’s being said is entirely different. Similarly, “deepfake” visuals are a reality. Profitability or manipulation in all forms affect how we provide judicial reasoning, make loan decisions, determine policy recommendations, interpret scientific results, and process content online. We have come to rely on systems that may identify a love letter as a legal contract and automated translators that don’t understand the language they are translating. Isn’t this evidence of an excess faith in statistics for sensemaking? What we have is a capability trap, and we don’t know how to admit it even amid growing unease surrounding AI. Back in the 1970s, AI leaders Marvin Minsky and Seymour Papert (1971) discussed the split between refining technical capacity (the “power strategy”) and ways of calculating, classifying, interpreting, and using information (the “knowledge strategy”). In their words, this is “a more sophisticated kind of ‘trade-off’ that we do not yet know how to discuss.” Adapting Somerset Maugham’s thoughts on writing a good book, there are three rules to develop useful learning. But no one knows what they are. That’s because AI learning lacks a necessary counterpoint that informs human intelligence. Practical concepts can be made sense of in dualities: good and bad, rise and decay, charging and discharging, statics and dynamics, health and disease, liberalism and conservatism…. Be it for power or knowledge, the obsessive focus on learning in AI misses something. Why does an opposite for learning matter? Assigning dominance to one purpose—learning without a counterbalance—may be detrimental. Polarities need to be thought through and managed well. An opposite cannot simply be ignored. Barry Johnson (2014) uses the example of breathing to illustrate the importance of dualities. Inhaling delivers oxygen; exhaling flushes out carbon dioxide. These are positive effects. The negative result of too much inhaling at the expense of exhaling is excess carbon dioxide; greater emphasis on exhaling than on inhaling supplies too little oxygen. These breathing contrasts are coupled and cannot be ignored as they are tied to another chief polarity: life and death. In this framing, the antidote for all the learning-by-doing in AI is not learning-by-not-doing but rather unlearning-by-doing. The question is how to make Watson unlearn from input rather than just to exclude it. Until that is understood and addressed, all our efforts in deep learning—however much depth is claimed—might sound triumphant but are ultimately shallow. Any AI system that doesn’t take unlearning into account is hardly a revolution; it’s not even a reaction. One might argue that AI does unlearn all the time. For example, it will analyze lots of images and then make guesses about whether what it is seeing is a cat; if it isn’t a cat, it “unlearns” and tries again. Not quite. A starting point for serious AI would be to aim for informed unlearning: understand what unlearning is and should be, and how it could guide fruitful learning. Both biological and cultural evolution present unlearning as an activity of renewal and reinforcement. Consider ecdysis, the process key for reforming protective structures—snakes shed skin, penguins molt, and so on. The process is inconvenient but essential. Consider depression. In one sense, what previously motivated an individual doesn’t have the same effect. Components of cognitive behavioral therapy, a form of treatment for depression, center on unlearning certain thoughts, beliefs, and attitudes in favor of learning new coping mechanisms. These examples are representations of evolutionary fitness and readiness. Engineering reminds us that contrary concepts can coexist and be constructive. With all kinds of trade-offs, when has engineering design ever worked without an opposing force? Engineering advances through learning and unlearning, although only the learning components are emphasized. The result has, alas, led to a business and policy boom to create more “learning systems” that foster higher performance and quality. This idea is incomplete, but it is a common desire in manufacturing, education, and health care. A high-level demonstration of how learning interacts with unlearning comes from Japanese technology firms, which have “an almost fanatical devotion to learning,” as Ken-ichi Imai and colleagues (1985) point out. Epson, the firm recognized for its printers, is known for having a next-generation model—that’s at least 40 percent better—ready by the time a “new” model is launched (Imai et al. 1985, p. 346). This meant Epson needed a different kind of learning practice among its employees: to become effective generalists, they needed to gain and at once apply an engineering and business sense to the product. Epson complemented this by embracing polarities, approaching “a new product idea from two opposing points of view. One idea is pitted squarely against another even when developing the next generation model of a successful product already on the market,” as Imai and colleagues note. “This approach opens the door for unlearning to take place and helps to maximize flexibility within the development process” (Imai et al. 1985, p. 361). At Honda, unlearning is practiced through what’s called the “rugby approach” (Imai et al. 1985, p. 353). This is different from a relay, where product development proceeds in sequence, individuals are responsible only for their piece of work, and they transfer control to the next unit; the quality of work at one state depends on the quality in its previous state. In a rugby model, the whole team “runs” together, coordinating their actions to get the ball to the goal. The method produces vigorous unlearning to depart from the relay-like hierarchies of most businesses and gain new advantages. This unlearning also creates a robust learning environment, akin to the way evolutionary selection and variation work at many levels, from an individual’s competency to a team’s capability to a market-generated preference. Unlike Epson, though, where polarity was appreciated in advance, Honda had the choice of just modifying the current version of its Civic or building a wholly new concept. The latter would require unlearning of design practices that Honda had put in place. As Imai and colleagues (1985, p. 361) put it: “What used to work in the past is no longer valid, given the changes in the external environment. To adapt to these changes, the challenge is to retain some of the useful learning accumulated from the past and, at the same time, throw away that portion of learning which is no longer applicable.” This is routine unlearning, where previously learned habits passively fade away. New learning -replaces or refines what was learned earlier. But for AI, and all “learning organizations,” more is needed. Wiping is a form of unlearning that over time works in two ways: through the “push” or pressure (as from a federal directive) to cease an action, or the “pull” or motivation provided by new information (as in a different delivery method for a medical treatment). Both these approaches, according to Rosemary Rushmer and Huw Davies (2004, p. ii11), are “deliberate and directed attempts at wiping out past learning; one using force, the other appealing to persuasion based on convincing evidence.” A third approach is undirected and unpredictable, and is perhaps the most valuable. This so-called deep unlearning, write Rushmer and Davies, involves a “new way of being and understanding that reflects a radical break with the past. This can be triggered by a sudden action, comment, or event; a single moment in which our lives are changed forever. This can be experienced when we are suddenly confronted with a major and substantial gap between what we see or hear and how we believed the world to be.” Unlike passive or smooth unlearning, Rushmer and Davies (1984, p. ii11) add, “the unlearner falls fast, far, and hard. The person that lands at the bottom is never the same as the person that began the descent.” This is hard change, necessary change, and useful change, and it fundamentally alters every aspect of how we learn. Unlearning isn’t easy; it’s harder than learning. And more learning or abruptly ceasing to learn doesn’t mean unlearning is automatically happening. It’s a conscious practice in which we concurrently establish new connections as we relinquish old aspects. Try learning a new language, and the ones you already know keep interfering. Material insights for learning and unlearning could come from a venerable Japanese tradition, very different from making printers and cars. The Ise Jingu– shrine is about 2,000 years old. Every 20 years, continuing a practice initiated in 690 AD, people tear down the wooden shrine and rebuild it from scratch. The unique belief of this ritual called Shikinen Sengu is that “repeated rebuilding renders sanctuaries eternal.”[2] The 30-odd events involved in Shikinen Sengu consume eight years; timber preparation alone takes four years. Reporting on this esteemed custom, one writer noted: “The renewal of the buildings and of the treasures has been conducted in the same traditional way ever since the first -Shikinen Sengu had been performed 1300 years ago. Scientific developments make manual technology obsolete in some fields. However, by performing Shikinen Sengu, traditional technologies are preserved.” In a deeper sense, while Shikinen Sengu could be taken as a case study in cultural transmission across generations, it also serves as a stellar motivation for learning and unlearning. The periodic disassembly and reassembly of the shrine is not destruction or inefficiency; it’s a cultural process of renewal, one that might simultaneously privilege both knowledge and ignorance. Shikinen Sengu illustrates that unlearning is a trainable virtue. The process may be technically inconvenient but it is culturally essential. We are told that learning is limitless. Does this mean we should simply keep acquiring information without a conscious effort to remove and renew? And there’s a bigger challenge: how to overcome individual and institutional resistance to unlearning that promotes rigidity, complacency, and intransience. This is precisely a scenario where AI systems can provide an advantage by pairing deep learning with deep unlearning. Just as there’s no stalemate between light and darkness, there shouldn’t be tension between learning and unlearning. Only with that appreciation can the current artificial intelligence become a different AI: accountable intelligence. References Edahiro J. 2013. Rebuilding every 20 years renders sanctuaries eternal: The Sengū ceremony at Jingū shrine in Ise. Japan for Sustainability Newsletter 26. Imai K-I, Nonaka I, Takeuchi H. 1985. Managing the new product development process: How Japanese companies learn and unlearn. In: The Uneasy Alliance: Managing the Productivity-Technology Dilemma, eds Hayes R, Clark K, Lorenz C. Boston: Harvard Business School Press. Johnson B. 2014. Polarity Management: Identifying and Managing Unsolvable Problems. Amherst: HRD Press. Minsky M, Papert S. 1971. Progress Report on Artificial Intelligence, Dec 11. Available at https://web.media.mit.edu/~minsky/papers/PR1971.html. Nuwer R. 2014. This Japanese shrine has been torn down and rebuilt every 20 years for the past millennium. Smithsonian Magazine, Oct 4. Rouse W. 2017. The systems, man, and cybernetics of driver-less cars. IEEE Systems, Man, & Cybernetics Magazine 3(3):6–8. Rushmer R, Davies HTO. 2004. Unlearning in health care. BMJ Quality and Safety in Health Care 13(S2):ii10–ii15. Winner L. 1984. Mythinformation in the high-tech era. Bulletin of Science, Technology & Society 4(6):582–96. [1]Inspired by the name of this quarterly, this column reflects on the practices and uses of engineering and its influences as a cultural enterprise. A related discussion can be found in Rouse (2017), p. 8. [2] Quotations are from Edahiro (2013). Also discussed in Nuwer (2014). About the Author:Guru Madhavan is the Norman R. Augustine Senior Scholar and director of NAE programs.