In This Issue
The Bridge: 50th Anniversary Issue
January 7, 2021 Volume 50 Issue S
This special issue celebrates the 50th year of publication of the NAE’s flagship quarterly with 50 essays looking forward to the next 50 years of innovation in engineering. How will engineering contribute in areas as diverse as space travel, fashion, lasers, solar energy, peace, vaccine development, and equity? The diverse authors and topics give readers much to think about!

Virtual Reality 2070: Vision and Challenges

Monday, March 8, 2021

Author: Carolina Cruz-Neira

Virtual reality (VR) can be defined as the spectrum of technologies that enable a computer-mediated reality, ranging from an enhanced real world (augmented reality) to completely digital worlds. Over the coming decades VR will capitalize on the convergence of 21st century technologies and scientific advances in areas such as artificial intelligence (AI), networking (5G and beyond), advanced computing, and IoT technologies to transform how people work, think, communicate, and enjoy life. VR will provide digital ecosystems (such as digital twins) that mimic physical ecosystems and will make it possible for people to think and operate, feel and interact with others, and have a full life parallel to that of physical reality.

Furthermore, because this parallel reality does not have to conform to the laws of physics, people can create new ecosystems and worlds with organisms, technologies, and rules not possible in a physical form. This presents both an exciting opportunity and some challenges.

Possible Applications of Virtual Reality

VR can bring tremendous benefits in a wide range of aspects of life and society (figure 1), as in the following scenarios:

  • Using available computing power and access to vast volumes of data, scientists can create virtual laboratories, free of the limitations of physical time and matter, to accelerate medical discoveries like drugs to prevent and cure diseases.
  • With AI and machine learning technologies, educators can create individualized virtual learning environments, in which AI-guided tutors deliver materials at a pace and depth matching the student’s knowledge level and learning preferences.
  • With simulations, data sciences, and sophisticated rendering methods, engineers can explore design alternatives for a new concept and then develop, test, and plan its production virtually before proceeding to a physical product.
  • Integrating imaging data, tests, history, and other data for each patient, doctors can use virtual bodies to plan customized procedures and treatments.
  • Technologies such as Lidar, 360 video, aerial -photography, and others make it possible to virtually visit the most remote corners of the world—or travel in time to explore places in the past.

Virtual reality will also enable new paradigms of social interaction, as it allows people to define a digital persona that may or may not be the same as who they are in the physical world—for example, with a different gender, race, or cultural background. It can thus enable people to experience the world and sociocultural situations from the point of view of others. This may enhance understanding and empathy.

Cruz-Neira figure 1.gif

VR 2020 versus VR 2070

VR has evolved from a limited, complex, and expensive technology in advanced academic laboratories or R&D industrial facilities to an affordable, simple consumer technology. But there is also limited diversity in platforms and VR is reduced to almost exclusively headset-mediated experiences.

Expanded Experiences

A first step to realize the potential of VR is to detach the current understanding of what VR is from the specific implementation platform. For instance, VR in the house or office may mean that the walls and other surfaces become the platform for immersion in a digital reality through embedded displays or projections. LED wall technology is rapidly evolving to be robust and to have enough resolution to create such walls, and projectors are starting to have ultralow throw distances that may make it possible to embed them in certain areas of offices and homes. These developments will better support teamwork and social interaction than headsets as users’ physical bodies are blended into the virtual space.

A more controversial approach may be the use of neural implants that send information directly to the brain, making it possible to alter physical reality at will. And newer network technologies, like 5G and beyond, may enable the wireless transmission of data and computation results to VR displays, making VR much more pervasive.

In addition, VR 2020 will be not only a visual technology but one that engages all the senses, with aural, olfactory, haptic (touch and sense of force), and even taste displays. Today the technological advances in those areas lag behind the fast development of visual displays.

Need for Guidelines and Standards

Another step toward VR 2070 will be the development of guidelines and standards for both hardware and software. At present there is no universal interaction standard to ensure that technical performance parameters and the ways virtual worlds are manipulated are consistent no matter the hardware and software used to create that virtual space.

There is also no universal content development environment in which VR applications can quickly be prototyped, developed, tested, and deployed. Each application is developed almost from scratch, without consideration of compatibility across platforms. The visual content of some applications is beautiful and artistic, others more cartoonish and less detailed. Some have AI-controlled elements, others have scripted behaviors. Some are single user, others collaborative and remote. Some include intense computational models, others simplistic approximations.

Moreover, models of interaction in VR applications are widely variable. It is nearly impossible to be functional in VR unless the developers of each application provide specific instructions about which buttons to press, where to look in the VR app, and what parts of the virtual world are “interactable.”

There are no guidelines on what level of “realism” is appropriate (including the need for sensory displays beyond visual) for VR applications to be acceptable and useful in a particular context. The considerable variation in the quality, details, and realism of a VR experience seems to be driven by individual developers’ skills, knowledge, and resources, not based on any guidelines on what makes VR applications effective.

All this creates confusion among VR consumers. Some may love VR if their first experience is with a good app, others may hate it if it is a poorly designed app. Of greater concern, a badly developed VR system and app can cause physical and mental problems, motion sickness, confusion, anxiety, even depression, which can have serious repercussions for users.

Making VR 2070 a Reality

The definition—and enforcement—of standards is the biggest challenge to the broader use of VR by 2070.

If this and other challenges can be addressed, I believe that the limits of what is possible with VR will be the limits of human imagination and ingenuity. This makes the 21st century one of the most exciting times to be an engineer!

About the Author:Carolina Cruz-Neira (NAE) is the Agere Chair Professor of Computer Science at the University of Central Florida.