Technological developments, such as social media platforms, have the opportunity to reduce social isolation. However, the opposite effects have been observed. For example, increased feelings of social isolation have been associated with increased social media use (Primack et al., 2017). Furthermore, there is a general trend for organizations to automate user experiences (Wright & Schulz, 2018) using technology like artificial intelligence, machine learning, and predictive analytics. These observations raise the question: how can we create user experiences that increase closeness between individuals, rather than increase social isolation? This is especially relevant as newly immersive technologies, such as mixed reality, become increasingly available to the public, since these technologies have both the ability increase isolationism and the ability to facilitate enhanced interpersonal communication. 

To understand how one might design for closeness, we turned to existing literature on facilitating and measuring closeness. There were indeed relevant instruments, such as scales to assess the closeness of interpersonal relationships (e.g., Berscheid et al., 1989), and measure of inclusion of the other in the self (Aron et al., 1992). There was also a rich ethnographic literature on how transnational families leverage technology to maintain emotional ties (e.g., King-O’Riain, 2015, Madianou, 2016), and how home users of video- mediated communication (VMC; Kirk et al., 2010) mediate closeness. However, we found few studies examining the qualitative experience of closeness from a multisensory perspective, especially as mediated by technology. Such insights will be important for developing design patterns for sensor-rich technologies, like mixed reality. 

To investigate the antipodal effects on social technology usage through a multisensory lens, we began with a fundamental set of observations into how individuals foster closeness using VMC. To form our foundational understanding of technologically-mediated closeness in complex emotional relationships, we believed that ethnographic data would capture the necessary intricacies that other methodologies may overlook. To this end, we conducted a rapid ethnographic study with five romantically-involved couples in long-distance relationships while the pair engaged in a controlled video call. Sessions were one hour in length. One partner came onsite to our offices for the study, while the other partner was remote, in a location of their choosing. Partners were provided with three neutral conversation topics, which they discussed for a maximum of five minutes each. Between each conversation topic, we systematically varied the amount of visual information that the remote partner received about their virtual partner (facial expression content; face and body expression content; face, body expression content, environmental context), counterbalanced across participant pairs. After each topic, participants completed a survey in which they provided information on feelings of immersion, closeness, agency, and other Likert scale vectors taken from mixed reality research literature on virtual presence (e.g., Slater et al., 2008, 2009; Waltemate et al., 2018). At the end of each session, we conducted a semi-structured interview, discussing the experience of the conversation and comparing the controlled experience with the couples’ interactions outside of the research setting. 

From the survey and interview data, four core vectors emerged, upon which closeness appears to be mediated: content (e.g., visibility of non-verbal cues), situation (e.g., comfort level in the given environment), identity (e.g., representation of the self in a digital medium), and agency (e.g., the agency to navigate between proxemic zones). We investigated the specific factors that contribute to the experience of each of these “closeness vectors.” Additionally, we explored how these vectors may work together to increase or decrease closeness, drawing on past research on multisensory perception and cognition. Furthermore, we posited sensory design principles for facilitating closeness in designs for new technology, such as mixed reality, which augments our perceptual and cognitive capacities with sensors and computing power.