Back to the future of virtual reality

A brief history of virtual and augmented realities

The new film Ready Player One provides a glimpse into a futuristic concept of immersive virtual reality. Set in 2045, the movie tells the story of a hidden game within a connected and interactive virtual reality platform in which characters can meet to escape from the hardship of their real-life city slums. While this may not be our experience yet, it is not far removed from the visions of the first pioneers in virtual reality.

VR used for medical purposes
VR and AR are being increasingly used in the medical and healthcare fields

While the term "virtual reality" did not enter mainstream discourse until the mid-1980s, the concept already existed in popular imagination. Initial exploration into 3D and a 360° field of view, simulated experiences, virtual worlds and head-mounted displays (HMD) had emerged by the late 1990s. However, it was not until the 21st century that the virtual reality experience became more established with the advent of smart phones, powerful computer graphics, motion controllers and computer interfaces that track gestures.

The theoretical beginnings of VR

Virtual reality began with works of fiction. Writers transported their characters to imaginary worlds that, based on today’s technology, appear to be prophetic. In one of the first short stories to imagine a virtual reality, Pygmalion’s Spectacles, published in 1935, characters wear goggles to experience "a movie that gives one sight and sound…taste, smell, even touch. […] You are in the story, you speak to the shadows, and the shadows reply, and instead of being on a screen, the story is all about you, and you are in it".

Other classics that stirred popular imagination included the television series Welt am Draht (1973) directed by Rainer Fassbinder, the movie Tron (1982) which transports a computer programmer inside a mainframe computer, The Lawnmower Man (1992) and The Matrix (1999).

However, it was the essay The Ultimate Display, written by Professor Ivan Sutherland in 1965, that provided one of the first scientific visions of virtual reality. It describes an "ultimate display" connected to a computer to create a virtual world that provides audible and tactile feedback and the ability to interact with objects. Many of the hypothetical suggestions made in the essay have since come to fruition, including gestural interfaces, eye-tracking, haptics, augmented reality and voice recognition.

Development of VR technologies

Initial attempts to create a virtual reality were based on photography and film. With the stereoscope developed by Charles Wheatstone in 1838, a viewer could perceive a 3D object by having each eye look at different 2D images. This technology remains relevant for augmented reality applications such as Google Cardboard which superimposes stereoscopic images onto a smartphone screen.

In the late 1950s, the cinematographer Morton Heilig developed the Sensorama, which sought to fully immerse the viewer in a short, multisensory film. A mechanical device that resembled an arcade game, the Sensorama provided the viewer with moving, 3D images, scents, vibrations and tactile sensations such as wind.

By the late 1960s, it had become possible to integrate electronics into virtual reality environments. In 1968, Ivan Sutherland, with his student Bob Sproull, developed the "Sword of Damocles" which is now considered to be the first HMD system. While the user interface and images were primitive, the system displayed output from a computer programme and could track the user’s eyes and head position.

The US government, through its Department of Defense, National Science Foundation and National Aeronautics and Space Administration (NASA), has funded many projects essential to the development of virtual reality technologies. Areas of research have included computer graphics, networked environments and simulation.

Edward Link developed one of the first commercial flight simulators in 1929 which was later used extensively by the US military to train its pilots during World War II. While the first flight simulators relied upon mechanical systems to provide pilots with feedback, the introduction of electronic systems greatly enhanced the user’s sense of reality. To train soldiers in dangerous situations, Charles Comeau and James Bryan developed the Headsight in 1961. This linked a remote camera to the movement of the user’s head.

In the mid-1980s, the engineer Thomas Furness designed a helmet for military pilots that projected information in an immersive and 3D space that pilots could view and hear in real time. His work, which began in the 1960s, included motion tracking, 3D sound, and the use of speech and gesture as sources for user input. To further increase the sensory environment for pilots, Louis Rosenberg developed the virtual fixture platform in 1992 which provides 3D immersive reality.

Haptic feedback began with the use of telerobotics by NASA and the nuclear industry. A master arm controlled a remote – slave – arm, usually in order to operate a space vehicle or handle irradiated material. The haptic systems developed by the University of North Carolina at Chapel Hill, under the auspices of the GROPE project, have also provided an important insight into the use of master control arms for virtual reality applications.

Consumer interest

The video gaming industry has helped to propel the technologies used for virtual reality and inspire popular imagination. Consumers, buoyed by visions of virtual reality from books and movies, began accessing technologies in the 1980s and 1990s. Gear developed to provide haptic feedback became commercially available from the company VPL Research, whose team included such VR pioneers as Jaron Lanier and Tom Zimmerman. Its equipment included the Data Glove, EyePhone and Data Suit.

By the 1990s, the gaming industry had begun offering virtual reality headsets and games such as Sega’s VR headset and Genesis console and Nintendo’s Virtual Boy. The company Virtuality developed a network of arcade machines that allowed gaming in a multiplayer immersive environment. However, by the late 1990s, the hype surrounding virtual reality had faded into disappointment and mass consumer demand diminished. Despite these setbacks, virtual reality continued to prosper within the gaming industry with the establishment of virtual communities and massively multiplayer, online role playing games (MMORG).

Current VR applications

Virtual and augmented reality applications are now flourishing beyond the gaming or home entertainment industries. From education and training to manufacturing and health care, these applications are becoming increasingly prevalent.

In education, students can interact with objects within a 3D environment. Medical students can be trained while watching live-streamed, 3D surgeries from anywhere in the world. Visitors can walk virtually through heritage sites or, in the case of the Kremer Museum, visit a virtual museum collection. First aid responders are immersed in seemingly real-life disaster scenarios while miners learn to recognize risks while walking through virtual mines.

Manufacturers benefit from virtual and augmented reality applications. Interactive 3D modelling tools used by car and rail manufacturers allow designers to view and test their products before production begins. HMDs provide employees with additional sources of data that are useful for inventory management or warehouse navigation. Construction crews can gather data to visualize real-time conditions and make adjustments as necessary.

In healthcare, virtual reality applications provide therapy to children with autism and help treat post-traumatic stress disorder. Surgeons use visual tools in preparation for an operation. Applications allow doctors to perceive the experiences of a patient with hearing and visual impairments.

IEC Standards

Virtual and augmented reality applications rely on such components as screens, processors, motion sensors, gyroscopes, cameras and images which are linked to hardware such as headsets. IEC technical committees (TCs) and subcommittees (SC) are responsible for producing International Standards that make the hardware and software used in these technologies possible.

The Joint Technical Committee of IEC and ISO, ISO/IEC JTC 1, provides standardization in information technology. Specifically, ISO/IEC JTC 1/SC 24 develops Standards relating to image processing, computer graphics and virtual reality.

IEC TC 47 and IEC SC 47F are responsible for the standardization of sensors and microelectromechanical systems. IEC TC 100 produces Standards which contribute to the quality and performance of audio, video and multimedia systems and equipment while IEC TC 110 covers electronic display devices including touchscreens.

What’s next?

While the movie Ready Player One gives us a glimpse of the possible future for virtual reality, that future remains elusive. More recently, the most successful applications have been able to provide an augmented reality, whether that’s finding Pokemons or tracking inventory. However, it is not yet clear how we will decide to merge virtual and real environments, much less why or which interfaces will be successfully adopted. Works of fiction may provide the best source of inspiration.