10. Supporting Disaster Preparedness Through User-Centred Interaction Design in Immersive Environments

In our contemporary world, climate disasters are having an increasingly catastrophic impact on communities. Yet, much focus of existing research and investment has been on “resilience”—broadly defined as the capacity to recover from an extreme event. While resilience is undoubtedly a crucial factor, of equal, if not greater, importance is “preparedness”. In 1735, Benjamin Franklin famously warned the people of Philadelphia, who were facing an unprecedented threat of fire, that “an ounce of prevention is worth a pound of cure” (Franklin, 1735). Being able to foresee a problem allows us to better prepare for it and reduce its negative consequences. At a time when wildfires and severe floods—that were previously classified as “one in 50-year events”—now occur annually, preparedness is at least as important as resilience. However, while research and investment into disaster resilience have grown over the years—including in interaction design (Satchell & Foth, 2011)—this is not the case for preparedness, where few innovative methods have been developed to improve the ways both the wider community and first responders can plan for extreme events. This gap in research is a catalyst for us to examine how user interactions in immersive virtual environments can support innovative and effective strategies for disaster preparedness.

The chapter commences with an examination of the role of intuition and experience in preparedness. It notes how being able to visualise, describe and communicate the experience of an extreme scenario allows people to better rehearse how they will respond to the real event. The chapter examines how the role of vicarious experience, developed using immersive 3D experiences, can be effective for (i) visualising and enhancing preparedness and (ii) developing narratives or stories to engage users. We then consider the creation of scenarios to accommodate the visualising, describing and rehearsing of extreme event responses, along with the technology required to achieve this. The primary approaches described in this chapter use augmented reality (AR), virtual reality (VR) and serious games (SGs). These have the potential to foster increased situational awareness and risk perception within communities—from first responders to local populations. We conclude with a discussion of future research required to further improve immersive environments for disaster preparedness.

Preparedness is a function of prior experience and of perceived threat (Lazo et al., 2015). While direct experience can be a strong motivator for enhancing one’s preparedness, the majority of people will likely encounter major extreme events (e.g. a large earthquake) without such prior exposure. They will need to rely on experience of smaller events (e.g. earth tremors), different types of disasters (e.g. flash floods), other adverse life events (e.g. accidents) or vicarious experiences (e.g. media reports or accounts of prior events by people affected) (Becker et al., 2017). Research found that such experiences inform people’s understandings and actions in relation to disasters (ibid.). Furthermore, the work of Klein (1998) and others into the experiences of first responders and disaster experts has shown that prior participation in similar events can activate intuition in a stressful situation that requires a time-critical response. It allows fast and generally accurate decision making (Miller, 2018). Professionals trained in such a way can draw on both “on-the-ground” experience and learned expertise to adapt to extreme situations (ibid.).

However, professionally trained first responders are no longer the dominant support force during extreme events. Fires and floods have become common in remote, peri-urban and suburban areas, where neighbours often attempt to help each other out of danger (Shaw et al., 2013). Members of the public often do not have relevant prior experience or expertise to draw on and often lack a rehearsed plan. They tend to receive only general instructions and advice in time-critical and stressful situations (e.g. through the media or emergency services broadcasts). Becker et al. (2017) found that the more direct peoples’ experiences of a disaster were, the more likely they were to relate to it in active terms (i.e. display threat awareness, demonstrate actionable knowledge and engage in interpersonal discussion). Equally, people would show a heightened understanding of its consequences and use future disasters as a frame of reference, form adequate beliefs and emotional connection and have a stronger motivation to prepare. Importantly, Becker et al. (2017) found that vicarious experience does not achieve the same level of personal hazard experience as direct or indirect experience of an event. As it would be unethical to subject people to extreme and dangerous events in order to equip them with first-hand experience, other means of fostering productive exposure are required. A way to provide a safe and powerful vicarious experience is to develop immersive interactive virtual environments that can realistically simulate an event—promising to facilitate a more visceral experience by making the participation feel as if they are present in the event—than, for example, merely reading a story or watching a film.

Professional disaster response training commonly includes VR simulation training to facilitate vicarious experiences of various emergency situations (e.g. Hsu et al., 2013; Nakanishi et al., 2009). Such role-playing and similar exercises can develop the type of participatory involvement needed for building intuition that can support decision making in disaster situations (Miller, 2018). Hence, they have been an integral part of emergency services accreditation and training for many years (e.g. see Alexander, 2000). In domains such as teacher and medical training (e.g. Korucu-Kış, 2021, Stegmann et al. 2012), vicarious experiences have been shown to be as effective for developing the skills to respond to real patients or students as actual direct experience. For example, trainees may observe others undertake procedures on simulated patients, or they may reflect on critical incidents and dilemmas in online settings. Across the board, combining vicarious and “hands-on” experience was found to be the most effective approach (Stegmann et al. 2012).

Professional emergency response decision makers generally have extensive data at their disposal to support them in their practice. Yet, this data is often only available in impersonal numerical form, which complicates assessment and rapid conversion into practical situational understanding—limiting their ability to appropriately respond to time-sensitive incidents. Members of the public have access to less data and often no relevant experience that supports them in responding to the critical situations, so they have even less usable information in a crisis situation. For them, the chance to gain a vicarious experience of an extreme event could be lifesaving. However, offerings of immersive scenario-based interactive activities to the general public are extremely rare.¹ To facilitate vicarious experience for professional responders, volunteers and communities, a pool of relevant prior experience must be assembled, coded and presented so that people can draw on a library of possible actions when immersed in an extreme scenario—whether as residents, first responders or decision makers. For developing a VR training scenario, two methods are commonly used to present such experiences: scenario-based learning and serious games (SGs).

Alexander (2000) argues that scenario-based learning “is useful [for] developing skills [such as] time management, cognitive mapping, [mediation,] team management and decision making under stress”. He defined a scenario in the disaster/emergency education context “as a reconstruction of past [events] or a hypothetical construction of future ones” (ibid.). When used as a teaching or training tool, scenarios are often employed to stimulate discussion or reflection (Korucu-Kış, 2021). Instructional methods using scenarios typically involve assigning only a small number of roles to participants. However, if the number of roles and their complexity are broadened, an emergency simulation game can be developed (Alexander, 2000). There is evidence to suggest that games can be even more effective than scenarios when training young professionals in disaster scenarios. Ma et al. (2021) used a randomised controlled trial to compare “the impact of theme games and scenario simulation on the disaster nursing competence of nursing students”. They “found that the overall [scores for competence, cognition, skills and affective response] were significantly higher in the game-based teaching group than in the scenario-simulation teaching group” (ibid.). They attribute these findings to increased motivation and improved autonomy and feelings of competence for those in the game group.

A “serious game” is a game wherein the goal is knowledge development, typically to educate the “player”, rather than merely to entertain. SGs allow for agency: the capacity to shape the results, rather than just role-playing a set scenario. The player in a SG is not just experiencing a situation or scenario and reflecting on it—they are acting in it, trying out different options for response. One example of a SG for disaster preparedness is a game by Lovreglio et al. (2018). It uses VR and the gaming engine Unity to prepare people to evacuate a hospital in the event of an earthquake. The game environment was designed to allow interaction (with environmental features and non-player characters) and navigation through a real building (Auckland City Hospital). Lovreglio et al. (2018) note the complexity of simulating a natural disaster using just sound and vision, especially when movement (shaking of the ground), heat (from fire) or touch (water from sprinklers or flooding) is part of the actual experience. They also stress the importance of using “real world” settings and scenarios in SGs as a means of increasing engagement and learning. Tsai et al.’s (2020) research is part of a field known as “game-based disaster education”, which has a focus on preparedness and delivery “at scale”, meaning to large numbers of people in a community. They developed the game Battle of Flooding Protection to increase willingness and capacity to engage in disaster response activities. Tsai et al. (2020) argue that SGs are more engaging because they situate the participant in the scenario, allowing them to interact with and shape events alongside other people. SGs like these two examples typically award points or achievements to acknowledge successful outcomes or reward effective responses, activities or strategies. As such, SGs offer a means of engaging players in visceral immediate situations, potentially developing effective synthetic experience, which has the potential to fuel their intuition (Miller, 2018).

Not all scenario experiences need to be visually immersive to be effective. The examples presented thus far have been reliant on fully immersive VR experiences in labs or through head-up displays (HUDs). Yet, it is also possible to use AR (on phones with cardboard hoods and low-cost headphones) to create emergency scenario overlays of real communities. Such an approach may not have the level of technical finesse of the discussed SGs, but by using real-world locations that people are physically situated in, a different type of emotional engagement and interactivity can be achieved (Gonsalves et al., 2021). AR-enabled SGs prompt and facilitate the sharing of local knowledge among communities and provide a tool to rehearse localised disaster scenarios. Such techniques provide a useful co-design approach to rapidly iterating hyperlocal emergency response training. They may accompany and inform the design of more immersive narratives.

Tabletop AR sandboxes (see Fig. 10.1) are another approach to enabling more immersive encounters with scenarios such as intense floods. These devices use real sand as a tangible interface for participants to explore the movement of water through contours in a landscape (Sánchez et al., 2016). A colourful contour map is projected onto the sand, augmenting the physical sand with information to facilitate learning and exploration. Water is projected into deeper regions, mimicking the effects of fluid dynamics on the landscape. This approach allows participants to mould the sand into different heights and shapes, to observe the effect of terrain on shaping water movement and of rain in shaping terrain. AR sandbox installations invite easily accessible tactile participation and can support understandings of possible flood scenarios and geoscience more broadly. Sánchez et al. (2016) describe these experiences as being more intuitive and faster vehicles for community participation compared to computer simulations.

An A R sandbox presents the distribution of dunes in color gradient contours with shadows of people's hands. The shade of color turns lighter as we move outwards from the interiors.

Whether we develop scenarios (with or without reflective activities) or games or some combination of the two, designers need to consider at least four types of challenges: (i) understanding stories and contexts, (ii) creating scenarios or games, (iii) working with technology to deliver them and (iv) navigating equity of access.

A key consideration early on is how we garner or develop the scenarios so they are realistic enough to be both useful and engaging. Any design process must first seek to understand the nature and dynamics of the real-world scenario and its broader ecological, social, cultural, symbolic, spatial and temporal contexts. The designed scenario must be based upon the real-world conditions. We need to understand how people caught up in extreme events interact with the event, the available situational information, other affected people and life forms and surrounding artefacts (from their car to their phone, their hose or their house). Such information can be garnered through interviews with survivors and first responders; studying reports, books and media; and reviewing Google Maps or similar platforms. A useful skill is what Lertzman (2019) calls “radical listening”, which combines empathy, open-mindedness, presence and validation to conduct a deep and transformative form of listening that goes beyond the surface level.

Creative approaches to prototyping as well as community engagement throughout the design process have proven to be key to designing successful immersive experiences that support preparedness. Communities affected by disasters consist of different types of stakeholders with different levels of technical literacy and access to equipment. These range from professional disaster responders, State Emergency Service (SES) and fire brigade volunteers to local populations of all ages in various settings, from city centres to remote communities. Understanding who users are and designing for them is vitally important—especially since interaction needs to feel natural and intuitive, while the scenarios estrange familiar territory and depict novel and uncomfortable situations.

Participatory design approaches recognise different users as collaborators with their own context and domain expertise and actively involve them in the design process. Participatory design denotes a set of methodologies that reimagine traditional boundaries between designer and user. While precise definitions remain contested, researchers agree that a core component of participatory design is a process of collaboration in which mutual learning takes place (Greenbaum, 1991; Robertson & Simonsen, 2013). Co-design is typically viewed as a practice within the wider umbrella of participatory design (Sanders & Stappers, 2008). From this perspective, the design process becomes a space for mutual learning between collaborators, software engineers and decision makers, facilitated by the designer. The latter deploy their own expertise in facilitating experiences, drawing on a broad toolkit of methods (Kerr et al., 2023). Applying this approach to disaster preparedness can enrich the design process, incorporating local and tacit knowledge that the stakeholders have of potential disaster contexts. Active involvement is also proven to engender more widespread community support, agency and ownership and increases the likelihood of adoption of design solutions (Blomberg & Karasti, 2013). Dialogue in the design process is crucial when intervening in complex local decision making and environmental contexts for seeking to enhance disaster preparedness. SGs can be effectively integrated into a co-design process, where they can prompt idea generation and sharing of experiences and local knowledge.

One approach to better capture community insights is to engage participants in workshops creating tangible mock-ups. These are typically low-fidelity and tangible visualisations of contexts, leveraging readily available materials. Ehn and Kyng (1991) emphasise that the role of mock-ups lies not in their proximity to the real thing but in their ability to support communication, reflection and discussion between participants. This allows community members to take part in the making process and to communicate their knowledge at speed, without needing to understand complex technology nor to rely on software engineers to code their ideas. Mock-ups can be vehicles for mutual learning and communication. For Sanders (2002), the expertise of users (as experts of their own experience) emerges in what they “do”, “say” and “make”. While traditional user interviews may convey observable and audible knowledge, prototyping can be a tool to access tacit user knowledge.

Immersive experiences typically involve a spectrum of technologies, which Milgram and Kishino (1994) place on a “reality–virtuality continuum”. On this continuum, AR overlays real environments with digital information, mixed reality (MR) blends the digital and the real-world environment, and VR immerses users in a fully synthetic environment where the real world is no longer visible. Collectively, these approaches are often called extended reality (XR). Suh and Prophet (2018) add to this list non-immersive virtual reality, which captures users accessing VR experiences from a desktop computer with a two-dimensional interface. The spectrum does not necessarily entail a greater degree of immersion at either end. Rather, Suh and Prophet emphasise that different technologies offer different qualities of immersion. The appropriate technology should be selected based on a range of factors, including the affordances of the modality and how well-suited they are to the user’s specific needs and contexts.

Immersive technologies offer the potential to support preparedness rehearsal, simulating disaster events to the extent that users gain vicarious experience. For Ryan (2015) and others, immersion is not simply a property of the technology, but a perceptual experience engendered through interactivity and imagination. She posits immersion as an active process, which may be temporal, spatial or emotional, reflecting the importance of interactivity as a gateway to immersion. For Ryan, narrative immersion may also be evoked through a variety of texts, including books. A key goal and measure of immersive experiences is their ability to evoke a sense of presence. For Zhao (2003), the sense of “being there” (i.e. presence) is dependent on the users’ ability to suspend disbelief. Arguably, the ability of these immersive experiences to transport users is more closely tied to the interactive narrative potential of the experience than the degree of photorealism (IJsselsteijn & Riva, 2003; Seegert, 2009). Indeed, Seegert (2009) interprets presence as performance. Designers may support users to construct interactive narratives in which they are viscerally transported, as though they are truly present in disaster situations. While an experience does not necessarily need to be photorealistic to support feelings of presence, the experience should be credible in order to not break the imaginative spell, which sees the user suspend disbelief (Seegert, 2009). This may depend on the particular user. For example, firefighters confronting a fire scenario should be able to trust that the scenario they are envisaging is realistic and accurately depicted. While immersive technologies are not the only way to engender a sense of narrative transportation, they present opportunities to more fully envelop a user’s senses, as well as the opportunity to support highly interactive experiences through real-time game engines. Technologies such as XR and 360-degree cinematic environments may provide the opportunity to gain vicarious experiences of specific disaster scenario that would be too dangerous or difficult to replicate in the physical world.

Increasingly, experiences are not limited to one type of interface. New visualisation systems such as iFire (discussed by Del Favero et al in Ch. 2 of this volume) are examples of cross-reality (Maurer et al., 2022) immersive experiences designed to support firefighters and laypersons to rehearse encounters with extreme wildfires (see Fig. 10.2). iFire is powered by Unreal, a real-time game engine commonly used in virtual production. This technology enables experiences to be adapted and deployed across a range of visualisation platforms, including projection-based 360-degree and 130-degree 3D cinemas through to desktop screens and tablets—with capability to also run on head-mounted VR displays. When designing for such an ecosystem of modalities, designers must consider different contexts of use. For example, Del Favero et al. (2022) emphasise the affordances of room-scale spaces (such as 360-degree cinemas) for providing social perspectives, which inform and mediate the experience of all participants, as they interact not only with the virtual environments but each other as well. Similarly, immersive tabletop experiences may encourage this sort of real-world collaboration (Ens et al., 2021). Head-mounted displays may provide a more portable experience—while they retain immersion, they usually only do so on an individual basis. Laptops support a more accessible and affordable, if constrained, immersive experience yet still facilitate a sense of immersion by allowing the user to feel present through their navigational control inside a 3D environment (Del Favero et al. 2023).

An illustration of an iFire visualization system. It incorporates 180- and 360-degree projectors with people, a mobile phone, a tablet, a laptop, and a V R headset around a central database. All devices display fire simulations.

Engendering a comfortable user experience with what may be quite unfamiliar technology to users is a key challenge. iFire’s user groups are not necessarily passionate gamers or those familiar with VR. Within immersive cinematic environments, users interact with iFire via a tablet. Users can navigate through space and time to visualise environmental variables, such as wind, temperature and humidity, and to gain an understanding of their location in the virtual world. In future iterations, users will be able to adjust variables of the wildfire to generate plausible novel scenarios using the application’s inbuilt AI system. The advantage of the tablet as a more tangible user interface is that, compared to a gaming controller, a wider range of users are already familiar with the interface designs, since they are widely used in professional as well as private settings.

Research into intuitive interaction with interfaces and products over several decades has shown empirically that the ability to transfer prior experience to a new situation is the main factor that determines how fast and accurate a user adapts to a new interface or object (Blackler et al., 2010, Fischer et al., 2015, Still & Still 2019). For example, the use of touchscreens on smartphones or tablets may support a more intuitive user experience. By incorporating familiar and accessible components, these interfaces may support communities to more readily adapt to novel XR experiences. In order to tap into such prior familiarity, designers need to understand the level of familiarity of potential end users with the technologies they want to employ and adapt design plans accordingly.

While we see great potential and opportunities for the use of immersive technologies for disaster preparedness, specifically AR/VR applications, we must also consider critical issues that warrant attention and further research efforts. Several key concerns need to be addressed to ensure comprehensive and socially responsible approaches to the design of immersive environments for disaster preparedness.

First, it is paramount to keep the digital divide and digital inclusion issues front and centre (Dezuanni et al., 2018). Access to high-fidelity technology is not uniform, and there is a risk of exacerbating existing inequalities if community preparedness relies on technology that is only available to some stakeholders. Designs need to cater to different technical literacy levels and varying access to devices and account for the disparities in Internet infrastructure and available bandwidth—as this can hinder the widespread implementation and uptake of immersive technologies. This calls for investigating low-tech and low-fidelity technology alternatives (Gonsalves et al., 2021).

Second, designers need to consider not just urban and metropolitan contexts but also how technical solutions can be transferred to rural and remote communities with precarious Internet access. Research has started to explore the digital capabilities needed for building disaster resilience (Marshall et al., 2023). Failure to address this may prevent vulnerable populations from capitalising on the benefits of advanced disaster preparedness tools.

Third, disaster preparedness often heavily focuses on positivist data sources, neglecting the nuanced ways in which communities make sense of disasters. Similarly, the community learnings derived from participation in scenarios, AR/VR environments and SGs are of a qualitative nature and can clash with institutional expectations and statutory requirements of local disaster preparedness planning. Research that addresses these epistemological divides is needed to be able to better incorporate community learnings, Traditional Owner knowledge,² oral histories and storytelling into disaster planning—recognising the value of both approaches.

Fourth, design scholarship has long recognised the need to emphasise community-centric approaches rather than succumbing to technological solutionism (Morozov, 2014). While we already argued that employing participatory design and co-design is an effective path to capturing the needs and perspectives of end user communities, there remains a risk of governments co-opting technology for solely tokenistic purposes, as mere symbolic gestures or “engagement theatre” (Kamols et al., 2021; Foth et al., 2018). Design “institutioning” refers to an emerging recognition in design research that advocates for meaningful participation and engagement with institutional stakeholders in co-design processes, which can help designers work strategically with the community to guard against superficial uses of technology (Teli et al., 2022).

Designers able to critically examine these issues and successfully navigate equity of access, ability and affordability are more likely to develop designs that account for the social, economic and cultural dimensions of implementing immersive technologies for disaster preparedness.

This chapter has presented an overview of the ways interactive immersive 3D environments can transform current disaster preparedness practices. With a focus on dynamically evolving extreme events, such as wildfires and severe floods, this chapter argues for the importance of focus on both preparedness and resilience. However, preparedness has often received less attention than resilience, even though investing in preparedness may provide better outcomes for society (Das, 2018). One reason for this lack of attention is the complexity of preparing diverse stakeholder groups, from first responders to entire communities, for evolving, often dangerous situations (Kohn et al., 2013). Nevertheless, the examples canvassed in this chapter demonstrate that an effective broad-based disaster preparedness system can be created. A system with the capacity to immerse people in vicarious, safe, interactive spatial scenarios must be able to engage people in multiple ways, to heighten their experience and reinforce particular behaviours or strategies. SGs use rewards or ratings for this purpose. However, the most effective reinforcement learning approach is founded on personal fulfilment. This arises when a person feels an emotional connection to the people, places or values they are interacting with in the immersive scenario. Becker et al. (2017) emphasise the importance of emotion in effective experience and preparedness. An important caveat is that, as part of this process, we need to consider how we deal with vicarious or actual trauma, retraumatisation or anticipated potential trauma of users when tapping into previous experiences and in workshopping scenarios or games.

In this chapter, we discussed examples of several different, albeit connected approaches to achieving an effective disaster preparedness environment. From VR and AR to SGs and mock-ups, the chapter presented evidence of the efficacy of these approaches to preparedness and their flexibility and adaptability for accommodating diverse scenarios, narratives and means of visualising extreme events. While the examples in this chapter also reference a range of current technology, more recent developments, such as generative AI, smart glasses and sensory substitution wearables, clearly offer potential for new research into immersive, 3D learning environments that are needed for disaster preparedness. Sharma et al. (2019) demonstrate one way these advances can be leveraged to improve preparedness, incorporating AI agents to simulate complex social dynamics in a collaborative immersive emergency response training. The integration of these emerging techniques alongside VR, AR and SGs is starting to be explored by researchers, and their early findings, such as those in the cited iFire project, collectively suggest exciting new opportunities for the field.