Cinematic virtual reality

Cinematic virtual reality (Cine-VR) is an immersive experience where the audience can look around in 360 degrees while hearing spatialized audio specifically designed to reinforce the belief that the audience is actually in the virtual environment rather than watching it on a two-dimensional screen.[1] Cine-VR is different from traditional Virtual Reality which uses computer generated worlds and characters more akin to interactive gaming engines, while cine-VR uses live images captured thorough a camera which makes it more like film.[2]

When storytellers began working in cine-VR, they applied many of the same cinematic narrative rules, but the technology demonstrated that VR can offer different possibilities that go beyond "traditional" cinema which will require new techniques and practices.[3] Harrison Weber, journalist of Venturebeat, described cine-VR like this: "It's a lot like film, only it puts the audience inside your story. With it, you can create entire worlds for your audience but none of the original rules of cinema apply. How do you create your art when all of your tools have changed?"[4]

The Human Interface Technology (HIT) Lab at the University of Canterbury differentiates cine-VR from other content created with 360-degree cameras based on the content, likening the prefix "cinematic" to that of "narrative". The HIT Lab requires cine-VR to be "narrative-based, instead of purely for novelty, entertainment, exploration, etc."; the cine-VR experience can be a drama, a documentary, or a hybrid as long as the story contains a beginning, a middle, and an end.[5] According to Ohio University's Game Research and Immersive Design (GRID) Lab, a cine-VR project differentiates itself from 360-degree video by using cinematic production techniques such as lighting design, sound design, scenic design, and blocking techniques (the latter two in the case of dramatic work).[6]

The concepts of "immersion" and "presence" are central to cine-VR.[7] The term presence is defined as "a sense of being there"[8] and described as "a feeling of actually being on location in a story rather than experiencing it from the outside".[9] Scholar Christian Roth differentiates immersion from presence by defining immersion as an objective criterion which depends on hardware and software, while presence is defined as the more subjective, psychological sense of being in the environment, and mainly influenced by the content of that world (e.g. story, characters, and location).[10] Immersion could be seen as a quality of the medium, in this case a cine-VR experience, while presence is a characteristic of the user experience; hence, higher immersion may lead to or result in deeper presence.[11] Immersion has objective components that can be advanced by technical considerations like image quality and sound quality, while presence is affected by individual users' subjective variations but is aided by the technical aspects that foster immersion.[12]

Cine-VR provides a more photorealistic user experience than traditional virtual reality, but current technology does not allow the audience to move around in video. In some ways, cine-VR is a trade-off, as fully computer-generated VR looks less realistic than cine-VR but is more interactive.[12] The ability to look around inside of a virtual reality space is known as three degrees of freedom (3 DOF), while being able to move around inside a virtual environment is known as six degrees of freedom (6 DOF). 6 DOF should ideally enhance the user's sense of presence.[7] The 3 DOF of cine-VR are defined as yaw (rotating your head left or right), pitch (tilting your head up or down) and roll (tilting your head on its axis upside down or right-side up).[3] Since experiencing a story using 3 DOF is quite different from watching a traditional film, television show or stage play, storytellers have recognized a need to develop a new creative language for cine-VR.[13] The key element that differentiates cinema and cinematic VR is the new role of the audience. Technologically speaking, this requires the storyteller to embrace the concept of immersion.[3]

With the ability to use 3 DOF, the cine-VR audience can freely choose the viewing direction when they experience the story. Therefore, traditional filmmaking techniques for guiding the viewers' attention cannot be used: techniques such as panning the camera or cutting to a close-up shot are no longer available to the filmmaker; instead it is the viewer who decides where to look.[14] Subsequently, in cine-VR, the storyteller has to rely more on lighting, sound design, and how the characters and sets are arranged to best tell the story.[13] Famous filmmakers have been attempting to do this at least since the mid-2010s when Kathryn Bigelow directed the cine-VR piece The Protectors (2016), Doug Liman directed Invisible (2017) and Alejandro Gonzalez Iñárritu, debuted Carne y Arena / Flesh & Sand at the Cannes Film Festival in 2017.[2]

  1. ^ Mateer, John (2 January 2017). "Directing for Cinematic Virtual Reality: how the traditional film director's craft applies to immersive environments and notions of presence" (PDF). Journal of Media Practice. 18 (1): 14–25. doi:10.1080/14682753.2017.1305838. S2CID 149242071.
  2. ^ a b Ross, Miriam; Munt, Alex (1 June 2018). "Cinematic virtual reality: Towards the spatialized screenplay". Journal of Screenwriting. 9 (2): 191–209. doi:10.1386/josc.9.2.191_1. hdl:10453/127306. S2CID 191658470.
  3. ^ a b c Pérez, Júlia Martínez (2016). Cinematic Virtual Reality: A Definition and Classification Proposal (PDF) (Thesis). S2CID 54916725.
  4. ^ Harrison Weber. 2016. "How filmmakers are inventing the language of VR." https://venturebeat.com/2016/05/04/how-filmmakersare-inventing-the-language-of-vr/  Retrieved 2 January 2022.
  5. ^ Tong, Lingwei; Lindeman, Robert W.; Regenbrecht, Holger (18 May 2021). "Viewer's Role and Viewer Interaction in Cinematic Virtual Reality". Computers. 10 (5): 66. doi:10.3390/computers10050066.
  6. ^ Williams, Eric R.; Love, Carrie; Love, Matt; Durado, Adonis (2021). "Cine-VR: A new medium". Virtual Reality Cinema. pp. 1–19. doi:10.4324/9781003028284-1. ISBN 978-1-00-302828-4. S2CID 234089129.
  7. ^ a b Love, Matt; Linscott, Charles P. ("Chip") (2021). "Cine-VR Production Overview". The Power of Virtual Reality Cinema for Healthcare Training. pp. 11–20. doi:10.4324/9781003168683-2. ISBN 978-1-00-316868-3. S2CID 238655342.
  8. ^ Riva, Giuseppe; Mantovani, Fabrizia; Capideville, Claret Samantha; Preziosa, Alessandra; Morganti, Francesca; Villani, Daniela; Gaggioli, Andrea; Botella, Cristina; Alcañiz, Mariano (1 February 2007). "Affective Interactions Using Virtual Reality: The Link between Presence and Emotions". CyberPsychology & Behavior. 10 (1): 45–56. doi:10.1089/cpb.2006.9993. PMID 17305448.
  9. ^ Henrikson, Rorik; Araujo, Bruno; Chevalier, Fanny; Singh, Karan; Balakrishnan, Ravin (2016). "Multi-Device Storyboards for Cinematic Narratives in VR". Proceedings of the 29th Annual Symposium on User Interface Software and Technology (PDF). pp. 787–796. doi:10.1145/2984511.2984539. ISBN 978-1-4503-4189-9. S2CID 16214242.
  10. ^ Roth, P. C. H. (2016). Experiencing Interactive Storytelling (Thesis). hdl:1871/53840.
  11. ^ Vosmeer, Mirjam; Schouten, Ben (14 June 2017). "Project Orpheus A Research Study into 360° Cinematic VR". Proceedings of the 2017 ACM International Conference on Interactive Experiences for TV and Online Video. pp. 85–90. doi:10.1145/3077548.3077559. ISBN 978-1-4503-4529-3. S2CID 24856146.
  12. ^ a b "Immersion, Presence, and Reality Trade-Offs". The VR Book. 2015. pp. 45–52. doi:10.1145/2792790.2792796. ISBN 978-1-970001-12-9.
  13. ^ a b Kobylinski, Paweł; Pochwatko, Grzegorz (2020). "Detection of Strong and Weak Moments in Cinematic Virtual Reality Narration with the Use of 3D Eye Tracking" (PDF). ACHI 2020, The Thirteenth International Conference on Advances in Computer-Human Interactions. pp. 280–284. ISBN 978-1-61208-761-0.
  14. ^ Rothe, Sylvia; Buschek, Daniel; Hußmann, Heinrich (19 March 2019). "Guidance in Cinematic Virtual Reality-Taxonomy, Research Status and Challenges". Multimodal Technologies and Interaction. 3 (1): 19. doi:10.3390/mti3010019.