Download PDF Spring Bridge Issue on Engineering and Climate Change March 15, 2020 Volume 50 Issue 1 The seven articles in this issue cannot cover all engineering-related aspects of climate change, but they highlight several areas of concern. An Interview with . . . Amy LaViers Friday, March 13, 2020 Author: Amy LaViers RON LATANISION (RML): We’re delighted to talk with you today, Amy. Let’s start by learning a little about your interest in both engineering and dance. AMY LaVIERS: I always had an interest in both things. When I was looking at colleges, I looked for places that had a strong engineering program and places that had a strong dance program, never thinking I would combine the two interests the way that I have. I was just following the thread of what excited me and what I wanted to continue doing in college. Amy LaViers photo Then I got to study both things side by side. On the last Thursday of my junior year I had two classes: automatic control and a dance writing seminar. In the morning was the last lecture for automatic control; it was a notoriously hard class we all had to take—I was very confused in that class. That day the instructor showed us an MIT unrideable bicycle. Usually when you ride a bike, you turn the handlebar and the front wheel rotates; on this bike, they switched that and made the back wheel rotate from the front handlebars. This changes the system, making it take years to practice to learn to ride this bike. That example really resonated with me. All of a sudden, I completely understood this course that had been so mysterious to me: I could imagine a bike that I couldn’t balance on and the difference between an unstable position—when it’s related to my body—and a more stable one. That afternoon I went to dance writing class and we watched a video of Twyla Tharp, the famous choreographer who has worked with both modern and ballet dance styles. She talked in this interview about the stability of different home positions in dance. In ballet, you can put the heels together and point the toes out and it creates a very unstable moment, it’s a little wobbly. But if you need to lift one leg from that position, it doesn’t affect you as much as when you stand with your two feet parallel and right under your hip bones—that’s the home position in modern dance. It’s very stable and easy to stand like that, but if you need to pick up one foot, you disrupt that stability and have to shift your weight in order to create space to lift the foot. She talks about the inherent stabilities of these different home positions and how that leads to different patterns and movement. For me that was a moment of ‘oh my gosh, that’s what we’ve been studying in my control class.’ That’s when I decided I would love to use the mathematics I was learning in my engineering coursework to study different styles of motion. And that’s what I’m working on today, in the context of robotics. RML: Have you danced professionally or in a performance? DR. LaVIERS: I never know how to answer that question. I’ve been paid to perform, to dance, but I’ve paid much more for dance lessons than I’ve been paid to dance. In terms of how professional I am, I think that’s the best way I can explain it. As part of my research, group members and collaborators perform regularly, including on professional stages, such as the Dance Now Festival at Joe’s Pub at the Public Theater in New York City. Thus, in that sense, I still actively perform dance as part of my profession. Photo 2 CAMERON FLETCHER (CHF): When you say you’re performing on stages in New York, do you mean dancing with your body or having robots perform movements? DR. LaVIERS: Both. CHF: Together? DR. LaVIERS: Together. RML: What is an example? DR. LaVIERS: My lab has an artist in residence named Kate Ladenheim, and she has collaborated with us to create a pair of robotic angel wings that she wears onstage to perform—they’re a physical metaphor for the ways technology often requires women to perform an ideal of gender. Kate wears a machine and sort of becomes an on-stage cyborg, using the physical machine as a way to bring scenes of technology on stage and push against what the expectations might be of an angelic woman. It’s been a great project for my lab, in particular thinking about what should be the connection between the performer and the machine? We’ve been working on breath sensors that create both a conscious and unconscious boundary between the performer and the wings. Of course, there’s also the question of how we attach the wings to her body. There are a lot of technical challenges that come up in that artistic expression. CHF: I see that one of your research areas is security and defense. What kinds of work are you doing that are related to dance for security and defense? DR. LaVIERS: The broad goal of the lab is in thinking about how movement expresses information and in trying to create artificial systems that have more complex information-rich movement. In defense that idea applies in a few ways. For a DARPA project I worked on, we were thinking about a movement specification language that would be platform invariant, the idea that I could take one sequence of commands and apply it to a host of robots. One reason I think I convinced DARPA that dance is an important part of answering that question is through my study of the Laban/Bartenieff Movement System, a movement taxonomy that underpins the movement notation system called Labanotation. I see the Laban system as a way of understanding -people’s perceptual signposts for perceiving intent in movement. Imagine you have a corps of dancers on stage and there is the idea that they are all moving in unison. That’s a perceptual phenomenon that we all experience: “Look at that group of 30 distinct indi-viduals moving in unison.” In fact, never will you see 30 perfectly mechanically similar people on stage doing exactly the same thing. It just seems like they are doing the same thing. The idea of imitation or moving in unison or doing the same thing is a perceptual feature of people. How did those people get to the point where you think they are dancing in unison? They use strategies, choreographic taxonomies, body-based language, and years of training to get to that point and to change their motion such that it looks like it’s the same. We use that idea to think about what it means for two distinct robots to take the same movement command and do “the same thing.” It’s not actually possible, but perceptually it is—if people think it’s the same, then we are starting to align our movement taxonomy or our programming language to people’s perception of movement. In one experiment we have a group of people use a shorthand version of Labanotation to label a human movement phrase choreographed in response to stimuli. We video the sequence and use their labeling to create the movement with a robot. Then new participants, who were not part of the first part of the experiment, watch the original human and three distinct robots, a large two-arm Rethink Robotics Baxter robot that cannot translate or move in space, a small Softbank NAO humanoid, and a mobile KUKA youBot with one arm and a mobile base. We ask the participants, Are these robots doing the same thing as the original human? From this we have recently developed a teleoperation system for rapid response to dynamic unknown environments and offering operators joint-space control. As opposed to telling a robot at each joint “do this” or “do that,” we use the large gross movement ideas from the Laban system to quickly create many joint angles moving at once based on higher-level commands like “move forward.” Photo 3 RML: I’m beginning to understand that, from a military point of view, you can program robots to do a lot of things, such as patrol or identify targets, but they can’t do it with this ease of motion that you, and perhaps DARPA, think would be more useful. Is that a general framework for the interest of DARPA in this work? DR. LaVIERS: Yes, I think so. As part of the grant, I interviewed soldiers who work with PackBots. They said that some of the things they want to be able to do with the robot involve translating the base and moving and articulating the arm at the same time. This is a more complex movement. It would be helpful if a soldier could translate intent to a robot as easily as he could shout to another human counterpart how to do these movements. We’re working on creating a language set for generating movement rapidly on the fly, which is really important for use in unknown environments. The physical hardware can do it. It’s the interface between the human and the robot that can’t rapidly disseminate those commands in the same natural way. There’s also not a sense of the right way to take an old program from the ’80s and put it on a current robot. Computers struggle with that a bit, but there’s a lot more interoperability of programs on different pieces of computer hardware than with robots. As a broader, more foundational piece, there’s also thinking about what’s the right way to do that for articulated machines with distinct physical morphologies. RML: In your work with graduate students, do they typically have interests as broad as yours? Are they more computer science oriented? DR. LaVIERS: I try to have students from many ends of the spectrum present in the lab. Part of the way I do that is I offer independent studies and collaborations with students who are not in mechanical engineering. They may be in dance, or from kinesiology or neuroscience. For the students who are doing a thesis with me and graduating from my department, I think what makes them most successful is having a background of some kind in what I call intense movement or embodied investigation. We’ve had rock climbers, Frisbee players, runners, classically trained Indian dancers, and ballet dancers. These are people who have spent a lot of time with their body; they value its physical intelligence and the expertise needed to give a correction to somebody or to change the strategy you’re using to, say, reach up from one rock hold to the next. Those students do the best. Robotics is really popular and a lot of students want to work with the lab, but a lot of students are not the right fit to work with the lab. I show students initial semester-long projects, for example, or some way to try out the lab. For example, we have a weekly writing hour and a weekly movement hour in addition to our lab meetings that are unusual in my field. Those events can be challenging for students who have a very different idea about what engineering is than I have. I think it’s push and pull in terms of both recruiting students and making sure they understand how unusual the lab is when they join. CHF: What have you learned from your students who have the intensive experience of using their bodies in “nondance” ways like rock climbing and Frisbee? DR. LaVIERS: I work to have everyone realize that we are all creating and choreographing movement profiles every day—that is, in my view, a “baby form” of dancing. Even if we’re all sitting around a table, I’m moving in a way that’s designed to make you think I know what I’m talking about. That’s a small act of choreography and a small act of dance right there. We went to a rock climbing gym and a climber showed us how it’s done. There’s a lot of choreography there. For example, you try a series of hand holds and if one doesn’t work what’s the reason? It could be a bodily reason because your arm doesn’t reach far enough so you have to create a different path. That’s a form of dance, adjusting spatial pathway. Let’s say there’s a hand hold that’s sort of far away, and you use a mental image of “punching up” to it, adjusting the quality of your intent. That’s a movement strategy that a choreographer might ask a dancer to use to get the right texture in a moment or a certain part of a piece. This is the lens I see the world through. To me, all those things are based on changes in movement that are perceivable to other human beings. CHF: So for you it’s a continuum, it’s not a matter of separate categories of movement. DR. LaVIERS: It’s definitely not a matter of separate categories of movement. In fact, what’s so great about dance is that everyone has their own mental image of it. If you study dance at a university, those faculty—like choreographers in New York City—are trying to come up with new movement profiles that you would not associate with dance and that you’ve never seen on stage. I recently saw a piece by Kimberly Bartosik, a former Merce Cunningham dancer. She’s working with gyrating, very physical, heavy, quick movements that hit hard and aggressively. It seems painful. You watch the dancers and think, ‘How can they physically do it?’ She’s creating a new texture that people haven’t seen before. That’s what dance is. It’s not about selecting from a canon—“here are the dance moves and here are the nondance moves.” It’s “how do I take this body and create a new idea that people have not seen on stage?” Another example is Yvonne Rainer, one of the more famous postmodern choreographers. She put on stage something that looks so utterly pedestrian that people were shocked, they thought, ‘How can walking around on stage be an expression of art?’ But it turned out to be one of the greatest expressions of art. It’s about innovating and finding new movement profiles. You can also go to a class where they will teach you moves A, B, C, D, E, F. But dance as a field, dance as a cutting-edge intellectual pursuit, is not that. RML: When you watch a professional athlete, a lot of what they do—I don’t know if I would call it dance, but it certainly has dance characteristics. For example, if you watch the footwork involved for a first baseman, readying himself to catch the ball from an infielder, it’s like ballet. I think also of gymnastics. You’re probably closer to gymnastics with the kinds of things you could presumably prepare a robot to do as a means of examining gymnastics movement. It occurs to me that there are a lot of areas that your work could impact, such as medicine or health care. Movement for artificial limbs, for example. Have you thought about any of that in terms of the work you’re doing? DR. LaVIERS: I often tell my students that we all perceive pattern and motion. With these moments in baseball that you’re comparing to ballet, I think what you’re seeing is human grace and coordination and harmony and physical movement that has been practiced and perfected for a particular moment in a particular context and it just looks beautiful and right. RML: We don’t -necessarily think of athletics as being graceful, but it’s better if you can watch in slow motion. There seems to be a lot of dance and athleticism required to make a good first baseman. DR. LaVIERS: One question we’re thinking a lot about is what it means to look graceful. That’s a qualitative term that we apply to many different physical situations and I don’t know what defines it. Figuring it out involves asking questions about what looks graceful and what doesn’t. For an upcoming study we’re going to look at “robot movement A” and “robot movement B” and ask “Which of these is more graceful?” We also see grace in animals, in all sorts of natural creatures, but we rarely have quantitative models for what generated that movement—hence the advantage of using robots to study a question like this. I’m really interested in understanding what differentiates natural movement and artificial movement and understanding the benefits of both. In health care, that could translate into things like rehabilitation or having a better model for how people move under normative healthy circumstances. We also think about the caregiving setting; for one project we thought about having teams of robots care for people in their homes, particularly older adults. My lab’s contribution has been about ways to create systems whose movement changes in a new context, communicates an internal state. If you’re reaching for your reading glasses on a Saturday afternoon, the way you move to get them is different from running to get aspirin because someone is about to have a heart attack. We want to create robotic systems that are externally reflective of those very different internal states. Photo 4 RML: With your interest in computation and mechanical engineering and dance and so on, it seems like all the ingredients, all the infrastructure, to do important work on artificial limbs. DR. LaVIERS: We haven’t looked at artificial limbs yet, but a lot of the same ideas underpin that area. RML: Your undergraduate training was at Princeton, is that correct? DR. LaVIERS: Yes. I studied mechanical and aerospace engineering and dance. You can’t major in dance—to my parents’ great relief. But when I was there, there was a program in theater and dance; now dance is its own program and you can get a certificate, which is like the equivalent of a minor at Princeton. RML: Then you got your master’s and PhD in electrical and computer engineering at Georgia Tech. I don’t know the Laban/Bartenieff Institute of Movement Studies. Could you tell us a little about it? DR. LaVIERS: It’s based in New York City but they have programs all over the world that train certified movement analysts; I did my training in New York and Belgium. It’s a 2-year certification program that I did in modular chunks, like 2½ weeks at a time, over the course of 2 years. The Laban/Bartenieff Movement System is more like a dance degree than an engineering degree, but is probably somewhere between those two. It’s very analytical. For example, there are movement scales that traverse various Platonic solids, so you have to think about the progression and balance of those forms. As soon as I got my faculty position at UVA, I started that program because I knew it was an area that I’d been leveraging in my research but needed a deeper, proper exposure to. One of the areas we study is the Effort System, a way of categorizing different qualities of movement—a ‘flick’ versus a ‘punch’ versus a ‘slash.’ RML: Do you envision your life as that of both an academic and a dancer? How do you balance your two interests? DR. LaVIERS: I think my ultimate choice would be a joint appointment with a dance department and an engineering department. Both can be pursued with a university—or without a university. I don’t know what the ideal mix of activities is for myself, but for now the university’s a beautiful place to do this work. RML: How did you become associated with the DARPA programs? Did you respond to an RFP? DR. LaVIERS: I responded to their young faculty award (YFA) call in 2014 or 2015. As an assistant professor I had visited DARPA program managers who might be interested in my work and been encouraged to apply for the YFA. RML: Where would you say your work is at this point in terms of your own objectives and in terms of -DARPA’s interest? Have you had any of your products in the field? DR. LaVIERS: Not yet. But they renewed my grant for a third year, and I’m working on some follow-on projects to move toward understanding new adaptations. One of the ideas was to compare today’s robots and look for a natural correlate to the movement capacity of these machines from an information-theoretic point of view, rather than traditional measures of torque, force, speed, and precision. Through these lenses robots can outpace their natural counterparts. That has been true for many decades. But for something information rich and complex, there are a lot of ways that natural systems still outperform artificial systems. From very different viewpoints—not Newton’s viewpoint, I would say, but from a Shannon information theory viewpoint—I wondered what might be a natural correlate to robots. One program manager at DARPA is very interested in using this viewpoint to compare artificial and natural systems. For the natural correlate, it may be the tiny C. elegans worm. Quantitative models of its motion are richer, requiring more complexity, more numbers to describe a pose, than for typical modern robots. C. elegans has only 302 neurons and persists in dynamic unknown environments throughout its life. We are really curious about that. Why do C. elegans know how to do stuff that machines can’t do? RML: As you are speaking, another thought occurs to me. This year, we’re celebrating the 50th anniversary of The Bridge. When you look at the past 50 years, you can see just how amazing the transformation in engineering and technology has been in such a short period of time. The internet, robots, all the things that we take as a given today have emerged. Thinking about robots, their applications, the technology of building them, their capacity for movement, and other capacities, where do you think this field will be in the next 50 years? Do you have a vision of what the future will be for robots? DR. LaVIERS: Well, I’m a contrarian at heart. My answer to this question points in the direction opposite of where I think everyone else is. In my honest opinion, in 50 years we will have so much more respect for what natural systems can do. There are Boston Dynamics videos of a robot doing a back flip and people are going crazy about how this was solved. But that machine does a back flip off a box of a very particular height onto a surface with a very particular friction with a very particular lack of wind. I think in a few decades we’re going to have to come to grips with how incredible it is that humans can do back flips in so many environments—in the dark or in the light, in the rain, from a high point or a low point, on a full stomach or an empty stomach…. I think we underestimate the capacity for movement of natural systems because as mechanical engineers we’re fixated on measures of movement like force, torque, and precision. We see those as the way to measure movement. It’s true that, on those measures, robots outperform most natural systems, although there are some really interesting natural systems with crazy movement profiles. But the lens of dance encourages us to think about not just how well, how hard, how fast can you do this one thing but how many different things can you do and how much control do you have over every choice you make as you move through space. I think in a few decades we’ll understand the challenge of robotics better than we do today. I think there’s been a lot of overhyped promise from the technical community as well as media and related communities. Look at self-driving cars. I think the claims about what we can do in 5–10 years in that space have been irresponsible, coming from a lack of respect for how incredible nature is. These things go in swings. I think right now there’s a lack of deference to observation of natural systems, that I hope robotics will have in the coming decades. RML: I see the contrarian in your thinking, but I think you’re right. I think it’s good to balance all the pluses and minuses. CHF: Amy, it sounds like you’re forging this very interesting dynamic marriage of two fields that often are not put together. I’m wondering where you see your work going in say 10 or 20 years. What’s the natural extension of your thinking and exploration? DR. LaVIERS: That’s a good question and a harder one to answer. Some of that depends on what opportunities I’m given, being at that critical tenure moment. I think that could change a lot. Someone like me who doesn’t do the traditional disciplinary thing faces a special challenge in finding the right home for my work. I think of disciplines like lenses. If dance is blue and engineering is red and what I’m doing is purple, it’s never going to look as red to the red people and it’s never going to look as blue to the blue people as it could if it weren’t purple. What I’m curious about and want to be able to understand better in 10 years is this question of natural systems and their movement, where they may excel in ways we don’t completely understand. I’m fascinated by this. When you put a human body on stage next to a robot, that difference is highlighted even more because humans are so expressive. We’re trying to understand what that means—what is grace? What is expression? What does it mean to do the tango versus a pas de deux in ballet? They look different. Qualitatively, we can see that, but quantitatively modeling what is different about the two is the first question I’m interested in. I want to understand dance and how we move and how we change our profile. In the next 10 years, I might have 1 percent of that answer. CHF: The word that comes to mind as I listen to you describe your efforts and interests is groundbreaking. Have you talked with colleagues who are doing anything like this, aside from the people in your lab? DR. LaVIERS: A lot of people. There are many dancer-engineers and other artist-engineers, and part of my life is helping to amass this community of people who are thinking in a similar way. We do workshops at conferences or symposiums where we invite like-minded people. Some people who do very similar work are Ken Goldberg, Thecla Schiphorst, and Michael Neff. Among people my age, like Elizabeth Jochum, Kristin Carlson, Naomi Fitter, Kate Sicchio, Heather Knight, and Guy Hoffman, there are labs sprouting up that are centrally interested in how the performing arts can help robotics. CHF: Very cool. You are in on the ground floor. Good for you. RML: This has been a terrific conversation, Amy. Thank you for joining us this afternoon. I wish you all sorts of good luck in moving your work forward. I like what you’re doing, it’s very transformative. Congratulations and much good luck. DR. LaVIERS: Thank you both so much. Amy LaViers with a small humanoid NAO robot in her lab at the University of Virginia, 2014. Image by Stacey Evans. This conversation took place January 15, 2020. It has been edited for length and clarity. Kate Ladenheim performs “Babyface” at the 2019 Dance NOW Festival, Joe’s Pub, Public Theater, New York. The robotic wings, developed in residency with Dr. LaViers’ lab, create an onstage hyperfeminine cyborg character to explore the experience of the feminine gender in technology. Their motion and interaction modalities required students in the lab to think about -choreography and embodiment. Image by Yi-Chun Wu. Movement score notated in Motif, a shorthand notation system that uses symbols common to Labanotation, created by Amy LaViers. The score indicates four movements described through the lenses of spatial direction, movement quality, modes of shape change, and body landmarks. Dr. LaViers has also used Motif for robotic motion on a variety of robotic platforms using the same specification. Image by Amy LaViers. We are all creating and choreographing movement profiles every day. Catie Cuan (left) and Amy LaViers (right) perform “Trio,” an excerpt from Time to Compile, at the 2018 Dance NOW Festival, Joe’s Pub, Public Theater, New York; Ishaan Pakrasi (not shown) is operating the robot. Catie is an artist and graduate student in robotics at Stanford University and was the lab’s artist in residence 2017–18; Ishaan completed a master’s thesis in the lab at UIUC. The piece explored the feelings of frustration that occur both in programming a machine and in being a woman working in engineering. Image by Yi-Chun Wu. About the Author:Amy LaViers is Assistant Professor of Mechanical Science and Engineering, Robotics, Automation, and Dance Lab, University of Illinois at Urbana-Champaign.