From spatial to platial – the role and future of immersive technologies in the spatial sciences

Immersive technologies such as virtual and augmented reality have been part of the technology mindset in computer and geospatial sciences since early on. The promise of delivering realistic experiences to the human senses that are not bound by physical reality has inspired generations of scientists and entrepreneurs alike. However, the vision for immersive experiences has been in stark contrast to the technology that has yet to properly support that vision; the community has battled nuisances such as cybersickness, tethers, and display quality for the last few decades. With the "final wave" of immersive technologies, we are now able to fulfill a long-held promise and freely envision how immersive technologies change spatial sciences by creating embodied experiences for geospatial applications. These experiences are not restricted by time or place, nor are they limited to the physical world. This contribution envisions a future for spatial sciences that is enabled by immersive technologies discussing their potential and challenges.


Introduction and background
The societal relevance of immersive technologies is increasing rapidly. For decades, highend human-computer interaction labs and sophisticated training simulators have demonstrated the value of immersive technologies for basic research on humans' behavioral responses to environmental characteristics [5,68] and training environments, for example, in aviation and the military [45]. Current developments allow for immersive technologies to enter the mainstream and substantially expand their role in research, education, communication, and decision-making across the academic spectrum and a plethora of societally relevant issues. Immersive technologies can increase educational equity by providing access to high-end training or field sites at scale [28,40], they are natural environment for representing three-dimensional data [8], and are medium for visceral communication of climate change and its effects [23]. Immersive technologies have proven to facilitate learning languages [35], teach people how to make better choices in what they eat [10], and create empathy in people toward those less fortunate [1]. Immersive technologies are not a silver bullet to solve all of society's grand challenges, but they offer potential solutions worthy of exploring, especially for the spatial sciences.
"Immersive technologies" is an umbrella term for devices that deliver augmented, mixed, or virtual reality experiences. Researchers and industris may also use the terms XR, eXtended, or Cross-Realities when referring to immersive technologies (see Figure 1 for a visualization of [42]'s virtuality continuum). Augmented reality is defined as digital content superimposed onto the real world [62] [21], while virtual reality is defined as an experience without any access to our physical reality [4] [25]. Depending on the literature, mixed reality is either an advanced version of augmented reality that allows for tailored, three-dimensional integration of physical and virtual realities, or an umbrella term for all experiences that combine virtual and physical content [44] [42]. In case a virtual reality environment is experienced through a head-mounted display or cave, we use the term iVR to stress the immersive nature of the experience. There has been substantial debate and confusion about the terminology. I will briefly address some issues below, as clarifying any underlying concepts is essential for advancing the science of immersive experiences, including in the geospatial domain.

Critical concepts of immersive technologies and experiences
Immersion and presence are key terms to characterize immersive technologies and user experiences enabled by them. There is, however, unfortunate confusion regarding the meaning of these terms to the extent that not only different but opposite definitions exist [6,16,58,65]. It is insightful to look into the etymology of immersion as it reveals the conundrum (see below). The definition of the verb immerse shows that the original meaning of immersion describes an object plunged into a liquid. This sense of the term immersion is the basis for defining immersive technologies. It focuses on the hardware, which is a medium that caters to the human senses in the same way that our physical reality immerses the human body. Hence, a head-mounted display is more immersive than a desktop computer, and a system with a haptic suit is more immersive than one relying on hand controllers only.
(implied in immersed), from Latin immersus, past participle of immergere "to plunge in, dip into, sink, submerge" (see immersion). Figuratively, of study, work, passion, etc., from the 1660s. Related: Immersed; immersing; immersive. https://www.etymonline.com/word/immerse For the last 340 years, immersion has figuratively refered to the state of mind of the user, not the hardware. From this perspective, we can immerse ourselves in a book without becoming physically part of it. Many researchers have suggested using the term presence [58] or spatial presence for the user's response to the system and the design characteristics of the experience. Ijsselstein [24] saw presence as an experiential counterpart of immersion (see [64]), meaning that a system with highly immersive qualities will create a high sense of spatial presence, the feeling of "being there." The topic is complicated. Not only the system characteristics lead to a user's response, but several design characteristics of the entire immersive experience. And, not all aspects of the experience are exclusive to immersive technologies. Within the user-realm and their responses to an immersive system, we can define at least four essential concepts (see also [20,24,58,64]): • Sense of (spatial) presence refers to the "sense of being there" (an alternative term is telepresence). The idea of embodiment is closely related to this term [24,58]. • Co-presence or social presence describes the connection that a user may develop toward other characters (AI or human) and the social context of said experience [3]. • Flow describes a user's engagement with the mechanics, challenges, and rules of an experience [14,55]. • Narrative/sequential immersion refers to a user's investment in a story and their engagement with the sequence of events. This is similar to reading a book. In an immersive experience, the progression can be manifold, meaning it can use elements such as an evolving gameplay or the exploration of new places to keep the player engaged [20,48].
I am not advocating for a specific terminology for the concepts outlined above but have summarized them in Figure 2 as a schematic. Most researchers agree on the concepts but profoundly disagree on the terminology. It is important to note that there is a distinct lack of literature in the spatial sciences on these concepts and how they influence research, KLIPPEL education, and decision-making in geospatial applications. Not all aspects of immersive experiences (see Figure 2) are equally relevant to the spatial sciences because they originate in other disciplines, such as serious games and media effects [12,41,49]. However, it is essential to create an awareness of all aspects of immersive technologies and experiences to envision their future in the spatial sciences. Figure 2: A schematic that shows critical concepts of immersive experiences (see discussion above). There is an unfortunate abundance of terminological confusion. It is important to separate the characteristics of the system from the user responses to a combination of system characteristics and design choices. While there are unique characteristics of immersive technologies the user responses are not restricted to a specific system. They could, for example, describe a user's response to a traditional desktop VR experience, not only immersive VR experiences.

From spatial to platial
To fully embrace the paradigmatic changes that immersive technologies offer for spatial sciences, it is essential to revisit their affordances [19], which are directly related to the above concepts and terminological clarifications. The focus here will be on immersive technologies and not the broader concept of virtual reality.
Immersive technologies allow a user to become part of a representation and quite literally step into one's data [4,51]. The dichotomy that has existed ever since the first computers, of the user on one side and the system on the other, disappears in immersive technologies. Stepping into the representation itself paradigmatically changes we interact with data and subsequently our understanding of data. This transformative change happens on two levels. First, instead of representing data on two-dimensional surfaces, we provide a natural environment for three-dimensional data. Simpson [56] referred to this change as "3D-on-3D" compared to "3D-on-2D". Implications for understanding data are far-reaching, www.josis.org including representing data at scale and using motion parallax for depth cues [26]. Second, stepping into a representation also turns data into an embodied experience. Instead of communicating abstractly, we can access data representations through several senses and potentially create a more emotional response. We connect to data not only by visually interpreting their representations but also through a visceral experience that creates a sense of being there.
This sense of spatial presence is a distinct and obvious advantage for creating realistic representation of physical reality as immersive experiences can replicate (almost) all aspects of a place, which is one of the core concepts of the geo-spatial and earth sciences [13,53]. Psychologists have long been discussing the concept of psychological distance and how it challenges, for example, our ability to communicate climate change as a temporally and spatially remote event [59,60]. With immersive experiences, we can reduce the psychological distance to places, both physically and temporally, and create a platform for grounding understanding and decision-making in actual experiences. Scientists have been given a tool that allows them to viscerally communicate changes in natural and built environments and connect them to what a landscape, such as a forest could look and feel like under different climate change scenarios [23,54]; (see Figure 3 for an example).
Immersive experiences also change how users understand abstract data through an embodied experience [9,33,57]. Instead of indirectly manipulating objects smaller than the human body on computer screens, users are afforded a natural first-person perspective. This allows them to explore data in the same embodied way they would explore physical reality. The desktop metaphor is becoming substantially less abstract, and we are only at the beginning of fully comprehending the potential of immersive analytics [18,33,66].
Theoretical foundations for immersive experiences have been developed in several disciplines. For example, thinking of cognition as being grounded through embodied experiences has become an influential movement in the cognitive sciences [2,63]. Embodiment is also a critical component of prominent learning theories, such as constructivism [47,61]. Looking into the roots geospatial sciences, we intuitively understand the importance of an embodied, platial experience for both education and research [13,52]. The turn toward understanding the role of embodiment in human cognition has, in fact, contributed to the scientific foundations of the spatial sciences [32]. With immersive technologies that allow for an embodied experience of data from places in the past, present, and future, and that turn abstract data into place-like experiences, we are at a turning point for how we conceptualize research, education, communication, and decision-making for geospatial applications [15,22,30].

A tiny research agenda
The geospatial sciences are in an excellent position to embrace immersive technologies. For several years, the ability to efficiently create three-dimensional representations of physical environments, existing or planned, has evolved rapidly [7,11,[36][37][38]. We are witnessing tremendous developments in our ability to sense phenomena on earth through remote and photogrammetric approaches. We are also seeing modeling approaches that efficiently create three-dimensional data, turning abstract data spaces into potential candidates for visceral experiences [43]. Figure 3: Forests are increasingly under stress from biotic and abiotic disturbances, including insects, windthrow, and climate. Among them, wind is a major ecological disturbance. We used a spatially-explicit, forest simulation model (LANDIS-II) to simulate how windstorms and climate change affect forest succession in 50 years into the future and developed a procedural modeling based workflow to create visceral experiences of different scenarios [23].
However, as much excitement as there is surrounding immersive technologies, we can also state with certainty that simply putting on a headset to experience three-dimensional data is insufficient or even counterproductive. We have diverging empirical studies that have shown the advantages and disadvantages of immersive experiences [16,34,39,46]. Like any other communication medium in education, research, and decision-making, we need basic research that carefully examines the technology itself and its design choices. Comparable to the developing science around visualization, we need a science around www.josis.org immersive experiences that deconstructs its technology and design choices to establish evidence-based recommendations for improving immersive experiences.
On the technology side, we find questions that mainly address scalability issues. While immersive technology in general is becoming tremendously more accessible, it is still the case that not everyone will be accessing a high-end gaming computer; and this might not be a problem. Together with my team, we have conceptualized this aspect as SENSATIUM, the sensing-scalability continuum that captures the fact that an increase in sensing comes at a cost [29,31]. An increase in sensing allows for higher levels of interaction fidelity and ultimately a more natural user experience. In conjunction with other aspects of fidelity, such as the display, it is essential to address the system characteristics' effects on both subjective and objective performance measures.
Equally important are design choices made in immersive experiences. Design choices change the flow of an experience [27]; they affect the sense of presence either in the environment [67] or with other agents [50]. Is it, for example, sufficient to capture physical environments and replicate them in immersive experiences, or do we gain more desirable outcomes by gamifying the experience? Do immersive experiences indeed paradigmatically change how we communicate, educate, and make decisions about spatial environments? How do we establish a science of immersive experiences that guides both design and technological choices? How do we address critical questions tailored to the needs of geospatial sciences?
The appeal of immersive experiences, especially for geospatial sciences, has undoubtedly increased in light of COVID-19. With restrictions in place on social interaction, travel, and education, we need communication media that go beyond teleconferencing and provide means to maintain core aspects of human communication. Even disciplines such as the geosciences, which elevate the field to a defining concept of the discipline, are witnessing an opening of minds towards remote learning opportunities [17,30].
Immersive technologies will find their central place in the geospatial sciences because of their substantial appeal for communicating spatial data and information. While they have been discussed in several academic communities for decades, recent advancements will allow for widespread accessibility and a community large enough to empirically address central questions of how to design efficient and effective immersive experiences.