Methodology
The new virtual realities and the art of theater
The immersive entertainment industry includes a wide range of experiences, including immersive theater, virtual reality, escape room games and more. It has been valued at more than $60 billion in 2019. Even immersive theater, while representing only a small part of this industry, contributes more than $28 million to that value. However, immersive theater is gaining increasing popularity and attention due to its ability to engage audiences in a more direct and interactive way. The National Theatre of London is one of the leaders in integrating virtual reality with immersive theater, using this technology to create even more immersive and innovative theatrical experiences. For example, the National Theatre has created a show called “Draw Me Close,” which combines VR technology with live theatrical performance, allowing audience members to interact with a virtual actress and explore a virtual world while moving freely in physical space. This kind of integration of virtual reality with immersive theater could open up new avenues for the theater industry, creating even more immersive and innovative experiences for audiences. But what is immersive theater?
Is theater already an immersive art?
The methodology that CarraroLAB developed during the cickoff days of the SPORES project aims to understand how theater can already be an immersive art in itself. A technological implementation of theater must therefore begin with an understanding of those characteristics that bring it closer to virtual and augmented reality experiences. Amelia Stevens argues that what defines immersive theater as a style of theatrical performance is its ability to create intimacy through the storytelling elements used. In other words, immersive theater is distinguished by the way it actively engages the audience in storytelling, creating a more immersive and participatory theatrical experience. Stevens wants to achieve this through the use of different techniques, such as the use of unconventional spaces, physical participation of the audience in the performance, and interaction with the actors themselves. The goal is to create a more intimate and engaging theatrical experience that can leave a lasting impression on participants. This can happen especially by eliminating the frontality of the stage and thinking of the space as a place where the audience members are physically involved. In this way, theater would be truly “immersive” in that it would create a much more immersive and interactive theatrical experience than traditional theatrical performances, and its impact could be very powerful and lasting on participants. Indeed, it would seem that in immersive theater there is a tendency to eliminate the distance between audience and actors, bringing them together in the same physical space and thus breaking down the so-called “fourth wall” that traditionally separates actors from the audience. This would create a feeling of intimacy and active participation on the part of the audience, who would feel directly involved in the action and narrative. In addition, this breaking down of the fourth wall can create a sense of vulnerability for both actors and audience, who are both faced with the direct presence and interaction of the other. However, this vulnerability can also become a unique opportunity for connection, as viewers would feel more deeply involved in the story and characters, and actors would have the opportunity to interact with the audience in a more direct and authentic way.
From videogames to theater
One issue we wanted to discuss at the training days in Brescia was the tendency to merge theater with video games. This practice opens up new frontiers, but it also raises several issues. “Spectators” is a concept introduced by Italian Videogame Designer and Producer Fabio Viola, which merges the words “spectator” and “actor.” The term refers to the active nature of the audience in contemporary theater, where the boundary between those who watch and those who act on stage becomes increasingly blurred. According to Viola, viewers are not just passive observers of the media performance, but actors in their own right, as they actively participate in the video game narrative. The theatrical experience can learn from the video game by aiming to create an interaction between actors and spectators, in which both groups contribute to the creation of the story and narrative. This concept is based on the belief that theater should be a shared and interactive experience in which the audience can actively participate and in which actors can interact directly with the audience. In this way, viewers are not simply passive spectators, but become an integral part of the theatrical performance. Fabio Viola’s concept of “spectators,” among other things, fits into the trend of contemporary theater to move closer and closer to the audience and to break down the barriers between stage and audience, creating more engaging and interactive theatrical experiences. This trend is masterfully developed by Eugenio Barba’s Third Theater. The problem with theatrical shows that attempt gamification, in our opinion, is that they often tend to create examples that are less than models already present in the video game world.
Take for instance the Half Real project, an art installation that combines technology, art and music to create an immersive and interactive experience for the audience. The work was conceived and realised by Dutch artist Daan Roosegaarde, in collaboration with Italian musician Eelco Topper. The installation consists of a series of light pillars that react to the movement of the public through motion sensors. Visitors can move through the spaces created by the pillars and experience a feeling of immersion in an environment of light and sound. The installation was inspired by video game technology and the concept of ‘half-real’, which refers to the sensation of reality mixed with fiction that is experienced during play. The music and sound, created by Eelco Topper, were composed especially for the installation and integrate perfectly with the light and movement of the audience. Yet it is the actors themselves who claim that their play is inspired by the video game ‘Heavy Rain’, which allows for an infinitely greater range of interactions than the show.
Virtual reality as connection
Creating intimacy through a remote medium seems impractical, but during the trainings we brought several projects that would argue otherwise. Born out of a dystopian reality – the COVID-19 pandemic – Human Signs is an online participatory art project by Yuval Avital that to date involves more than 200 voice and gesture performers from 50 different countries. The project began with an artistic self-testimony recorded by Avital, in which he expressed in his voice all the feelings he felt during his isolation. The result was a mantra that, “replicating” the aesthetics of the virus, traveled around the world entering artists’ homes and inviting them to respond through voice and gesture, two of the most fundamental forms of human expression. Human Signs has steadily spread, engaging other artists and moving toward new forms of proliferation. The various artistic accounts are not only personal expressions of life in isolation during the pandemic, but are also part of a collective reflection on a unique historical moment; all voices and gestures become part of a living digital archive. Human Signs artists include: sound artists, including some of the most celebrated figures from the ancient tradition, the contemporary scene, and the experimental scene; religious performers of all faiths and key figures in classical and baroque music; dancers and soloists from leading contemporary dance and ballet companies; and choreographers and visionary performance artists from all over the world. Human Signs presented weekly live streaming of artistic testimonies in dialogue with each other and with the Human Signs mantra through Avital’s YouTube channel. The dialogues took the form of Ensembles and Constellations, offering complex dynamic compositions or static structures. The final form of Human Signs is the “website-as-a-museum” website.
Bringing a deep sense of togetherness, Human Signs aims to overcome physical distance and isolation. And in doing so, it invites us to explore spaces, languages and forms of interconnectedness in a different way, as part of a collective reflection on a unique historical time.
Virtual reality as an arena
In addition to the theme of togetherness, virtual reality can represent the exact opposite, creating highly competitive immersive places. “The Void”, for example, is a virtual reality amusement park that offers a fully immersive and interactive experience. The company was founded in 2015 and has since grown to become a popular destination for VR enthusiasts and thrill-seekers. ‘The Void’ experience combines VR technology with physical sets, props and special effects to create a unique and realistic environment. Guests wear a VR headset and a backpack containing a computer and haptic feedback system, which allows them to see and interact with the virtual world. ‘The Void’ is currently in several cities around the world, including Anaheim, Las Vegas, Dubai and Toronto. The company plans to expand to other locations in the future and continue pushing the boundaries of VR technology to create unforgettable experiences for its guests.
This example was used during the training to reflect not so much on the applicability of such a model (too expensive for SPORES), but on the energy and conflict potential of these new technologies.
Framing at 360 degrees
During the training course, the CarraroLAB team addressed the difference between media that use framing (cinema, photography, painting, classical theatre, etc.) and those that use space (cave paintings, frescoed rooms, virtual reality, Third Theatre, etc.). We used cinema as the most iconic example, in particular Steven Spielberg’s hostile reaction to 360-degree video. One of the first mainstream examples of 360 degree video is ‘The Battle of Fallujah’, a virtual reality documentary that puts the viewer in the middle of a firefight during the Second Battle of Fallujah, which took place in Iraq in 2004. The video was produced by the New York Times and published in 2016. It uses footage captured by embedded journalist Michael Kamber and photographer Louie Palu, who documented the battle and the experiences of the soldiers who fought in it. The video allows viewers to experience the intensity and chaos of the battle from a first-person perspective, with the ability to look around and see the action from different angles. The audio includes soldiers’ radio transmissions and the sounds of gunfire and explosions, creating a sense of immersion. The same year, The Guardian published ‘Displaced’, a two-part virtual reality documentary series telling the stories of refugees. The first episode, entitled ‘Displaced: Episode 1’, takes the viewer inside the Domiz refugee camp in northern Iraq, home to more than 30,000 Syrian refugees who have fled the ongoing civil war in their country. The video follows the daily lives of three families living in the camp, highlighting the challenges they face in finding food, shelter and safety, but also their hopes for the future. The second episode, entitled ‘Displaced: Episode 2’, focuses on the experiences of Rohingya refugees who have fled violence and persecution in Myanmar and are now living in refugee camps in neighbouring Bangladesh. The video tells the story of a Rohingya family and their journey from Myanmar to the refugee camp, as well as their struggles to adapt to their new environment and find a way to rebuild their lives. Both episodes use 360-degree video technology to immerse the viewer in the refugee camps and give an insight into what it is like to live as a refugee. The videos also include interviews with aid workers and refugee crisis experts to provide context and background information on the problems refugees face today.
The lesson to be learned is that 360 videos can be extremely engaging, both perceptually (The Battle of Fallujah) and emotionally (Displaced). Theatre can therefore use this medium as an immersion device, as will be the case in the play Elsewhere.
Theatre and virtual reality
The term ‘augmented reality’ was first used by Antonin Artaud, 20th century French playwright and theatre theorist, in his 1938 essay entitled ‘The Theatre and its Double’. In this text, Artaud describes a form of theatre that goes beyond realistic representation and seeks to create a kind of ‘psychic realism’, capable of involving the spectator on an emotional and sensory level. The author uses the term ‘virtual reality’ to refer to this theatrical experience, which is presented as a kind of simulation of real life, but which at the same time detaches itself from it to create a unique and immersive experience. In this sense, Artaud anticipates by many decades the contemporary concept of virtual reality, which uses advanced technologies to create an immersive and interactive experience that simulates reality, but is at the same time distinct from it. The concept of virtual reality, which has been developed and perfected thanks to technological advances in recent decades, would thus have originated in the world of theatre. However, the practical use of this concept in theatre is not comparable to other media, such as video games and cinema. However, CarraroLAB researched some particularly useful examples:
“Wonder.land” is a musical theatre production that premiered at the National Theatre in London in 2015. The production is a modern retelling of Lewis Carroll’s classic novel ‘Alice’s Adventures in Wonderland’. The story follows a young girl named Aly, who struggles to fit in at school and in her family life. One day she discovers a virtual world called ‘Wonder.land’, where she can create a new idealised version of herself and escape from her problems. The show features a mix of live performance and interactive digital elements, including a virtual world projected onto the stage. The music, composed by Damon Albarn, blends pop, rock and electronic influences to create a unique and contemporary sound. Despite mixed reactions, “Wonder.land” was considered an ambitious and courageous attempt to combine theatre and technology and remains an interesting example of how digital elements can be incorporated into live performances.
“Hamlet 360: Thy Father’s Spirit” is a virtual reality adaptation of William Shakespeare’s classic play “Hamlet”, produced by the Commonwealth Shakespeare Company in 2019. The production was created in collaboration with Google’s Creative Lab and showed the potential of virtual reality to create immersive theatre experiences. The play was filmed with 360-degree cameras, allowing viewers to move and watch in the virtual world as the play unfolds. The production uses innovative technology to bring the play to life, with a cast of actors acting in front of a green screen and the virtual world created through computer-generated images. ‘Hamlet 360: Thy Father’s Spirit’ was well received by audiences and critics, who praised the innovative use of virtual reality and the excellent performances by the cast. The production was considered an innovative example of how virtual reality can be used to enhance traditional forms of theatre and create new immersive experiences for the audience. Hamlet is also a vehicle for innovation in ‘VR + Theatre = YOU as Hamlet’, a unique theatrical production created by Corinna Di Niro, which combines virtual reality and live performance to offer audiences an immersive experience in the role of Hamlet’s protagonist in Shakespeare’s play. The production begins with audience members donning virtual reality visors and being transported into a digital recreation of Elsinore Castle, the setting of the play. Once inside the virtual world, the audience assumes the role of Hamlet and is guided through the story by flesh-and-blood actors physically present on stage. The flesh-and-blood actors interact with the audience, giving them cues and suggestions to guide them through the story and their performance. The audience is encouraged to fully immerse themselves in the experience, explore the virtual world and interact with the other characters in the show. The production is designed to be interactive and personalised so that each audience member can have a unique experience of playing Hamlet. The use of virtual reality technology allows the audience to fully engage with the story and experience the play from a completely new perspective. “VR + Theatre = YOU as Hamlet” makes innovative use of new technology and the immersive experience it offered. The production was considered an innovative example of how virtual reality can be used to enhance traditional forms of theatre and create new interactive experiences for the audience.
Spacetelling
The term ‘spacetelling’ was coined by CarraroLAB, in the course of its long experience in digital, to indicate a narrative methodology linked to space and place. It is to be understood as radically opposed to ‘storytelling’, precisely because it disrupts all the elements that distinguish it. To the linearity of the story is opposed the circularity of the exploratory space, to the chronological structure of a story, the aesthetic structure of a place.
With the advent of virtual and augmented reality, the time has come to take care of the relationship between people in space, and the organisation of space itself. Theatrical art, if it wants to fit successfully into the technological breakthrough of the new digital realities, must become (or return to being) the art of immersing and being immersed.