The scene is a crowded movie theater. On the big screen you see realistic dinosaurs rendered with advanced computer graphics. When they appear, your first responses are unexpected, involuntary, and quick. Your heart pounds, palms sweat, eyes open wide, body tilts backward, and your brain devises a plan to get up and run. The antidote to that discomfort is the familiar mantra – “Calm down, it's only a movie!”
Repeating “it's only a movie” works because it confirms that the raw pixels projected on the screen show things that are not actually there. The recognition that a picture is merely a depiction gives viewers room to appreciate and interpret the scene, and to consider the intentions of the filmmaker. Clark and Fischer (C&F) provide an excellent map, something missing in media psychology, of the dimensions of depiction and interpretation.
Realism
Much of the history of media technology is about inventions that promote natural responses. Bigger screens with higher resolution, virtual and augmented reality, computer graphics, three-dimensional sound, and better interactivity – all promote a sense of “being there.” And the inventions work. It matters, for example, whether you watch the dinosaurs on a smartphone, in 3D IMAX, or with VR goggles (e.g., Bailenson, Reference Bailenson2018; Reeves, Lang, Kim, & Tatar, Reference Reeves, Lang, Kim and Tatar1999). And the primitive responses influence thoughtful ones; they are memorable (Bolls, Lang, & Potter, Reference Bolls, Lang and Potter2001; Lang, Dhillon, & Dong, Reference Lang, Dhillon and Dong1995), positively evaluated (Bartsch, Kalch, & Oliver, Reference Bartsch, Kalch and Oliver2014), and the excitation often transfers to other contexts (Kramer, Guillory, & Hancock, Reference Kramer, Guillory and Hancock2014).
There are similar advances in robotic realism, including human-like skin textures (Hu & Hoffman, Reference Hu and Hoffman2019), more purposive uses of touching (Willemse & Van Erp, Reference Willemse and Van Erp2019), better body language (Marmpena, Lim, & Dahl, Reference Marmpena, Lim and Dahl2018), and more detailed facial expressions (Chen et al., Reference Chen, Hensel, Duan, Ince, Garrod, Beskow and Schyns2019). These features may give some social robots a commercial edge precisely because they make humans and machines less distinguishable. Even in an imagined Star Trek Holodeck future, it may still be possible to say “it's only a robot!” but increased realism nevertheless favors natural reactions.
Time domains
Depiction seems most relevant to longer time domains, and like for other media technologies when they were new, that domain is the current emphasis in robotics research. When people interpret, construe, imagine, or appreciate, this primarily involves “slow thinking” (Kahneman, Reference Kahneman2011), and C&F cite numerous good examples of how people reason about social robots in this time scale.
Media realism, however, causes quick responses that occur in seconds or less. “Fast thinking” research about social robots is relatively new but increasing. For example, when people touch a robot they show heightened arousal, similar to touching humans (Li, Ju, & Reeves, Reference Li, Ju and Reeves2017). And within seconds, people make judgments about the warmth and competence of social robots, just as they do for people in real life (Reeves, Hancock, & Liu, Reference Reeves, Hancock and Liu2020). Designers are focusing on other primitive features like eye contact (Kiilavuori, Sariola, Peltola, & Hietanen, Reference Kiilavuori, Sariola, Peltola and Hietanen2021), and how robots negotiate physical space (Hoffman & Ju, Reference Hoffman and Ju2014).
Discretionary framing
Media experiences are not only determined by media stimuli. People can choose a frame, at least temporarily. In one relevant research paradigm, people switch between interacting with media characters that they believe are either controlled by a computer or by another real person. The mere belief that people are interacting with a real person (and not with a character that only depicts someone real) results in greater arousal and better learning (Lim & Reeves, Reference Lim and Reeves2010; Okita, Bailenson, & Schwartz, Reference Okita, Bailenson and Schwartz2007).
It is also noteworthy that interventions designed to reduce the negative consequences of media often teach people, and particularly children, how primitive responses are triggered and how media professionals use them to control attention. This can reduce negative effects by teaching people to choose a symbolic experience rather than a natural one (Jeong, Cho, & Hwang, Reference Jeong, Cho and Hwang2012).
Sampling robots
There is likely far more variance between media characters (robots included) than variance between human responses to any one of them (Reeves, Yeykelis, & Cummings, Reference Reeves, Yeykelis and Cummings2016). Several years ago, we cataloged 342 social robots that were used in over 1,000 studies in the last decade (available at https://goo.gl/Gqpzkx). Variance between the robots is impressive and the catalog doesn't even include some of the most interesting current products (e.g., there are no experimental studies that use sex robots as stimulus material; Döring, Mohseni, & Walter, Reference Döring, Mohseni and Walter2020)
The point of the catalog is to show that any selection of a single or few robots can be easily biased. Consequently, stimulus sampling (Clark, Reference Clark1973; Cummings & Reeves, Reference Cummings, Reeves, Jussim, Krosnick and Stevens2022; Judd, Westfall, & Kenny, Reference Judd, Westfall and Kenny2012; Yarkoni, Reference Yarkoni2022) is critical for social robots. A discussion based on robots that are dinosaurs, sex companions, soccer competitors, or dance partners, will be different than one based on characters from Michelangelo and Shakespeare and the most common (and least exciting?) robots in health care and children's learning.
Conclusion
Symbolic and natural experiences are both important for understanding how media characters are experienced. One is not more important than the other, and one cannot be explained by the other. The switching that occurs between these frames offers a different answer to the author's social artifact puzzle about how people can think media characters are real, and at the same time realize that they are mere mechanical artifacts. They are both. And what matters most is how either experience works, which is applied when, and how they might interact over time.
You have
Access
The scene is a crowded movie theater. On the big screen you see realistic dinosaurs rendered with advanced computer graphics. When they appear, your first responses are unexpected, involuntary, and quick. Your heart pounds, palms sweat, eyes open wide, body tilts backward, and your brain devises a plan to get up and run. The antidote to that discomfort is the familiar mantra – “Calm down, it's only a movie!”
Repeating “it's only a movie” works because it confirms that the raw pixels projected on the screen show things that are not actually there. The recognition that a picture is merely a depiction gives viewers room to appreciate and interpret the scene, and to consider the intentions of the filmmaker. Clark and Fischer (C&F) provide an excellent map, something missing in media psychology, of the dimensions of depiction and interpretation.
The sweating, however, is different. This is a natural response, the main requirement for which is the mere recognition that there are dinosaurs on the screen (Worth & Gross, Reference Worth and Gross1974, Reference Worth and Gross2017). This and other primitive responses are unfiltered by any thoughtful mapping between real and virtual (Lang, Reference Lang2000), and they signal that the moment may require action rather than interpretation. Evolved over millennia, the responses are not needed to survive a modern movie, but they are nevertheless difficult to circumvent just because the symbols that roused them only mimic reality (Meshi, Tamir, & Heekeren, Reference Meshi, Tamir and Heekeren2015; Reeves & Nass, Reference Reeves and Nass1996; Shepard, Reference Shepard1990). Natural responses seem difficultly related to the concept of depiction, and especially to the elements of interpretation, imagination, and appreciation.
Realism
Much of the history of media technology is about inventions that promote natural responses. Bigger screens with higher resolution, virtual and augmented reality, computer graphics, three-dimensional sound, and better interactivity – all promote a sense of “being there.” And the inventions work. It matters, for example, whether you watch the dinosaurs on a smartphone, in 3D IMAX, or with VR goggles (e.g., Bailenson, Reference Bailenson2018; Reeves, Lang, Kim, & Tatar, Reference Reeves, Lang, Kim and Tatar1999). And the primitive responses influence thoughtful ones; they are memorable (Bolls, Lang, & Potter, Reference Bolls, Lang and Potter2001; Lang, Dhillon, & Dong, Reference Lang, Dhillon and Dong1995), positively evaluated (Bartsch, Kalch, & Oliver, Reference Bartsch, Kalch and Oliver2014), and the excitation often transfers to other contexts (Kramer, Guillory, & Hancock, Reference Kramer, Guillory and Hancock2014).
There are similar advances in robotic realism, including human-like skin textures (Hu & Hoffman, Reference Hu and Hoffman2019), more purposive uses of touching (Willemse & Van Erp, Reference Willemse and Van Erp2019), better body language (Marmpena, Lim, & Dahl, Reference Marmpena, Lim and Dahl2018), and more detailed facial expressions (Chen et al., Reference Chen, Hensel, Duan, Ince, Garrod, Beskow and Schyns2019). These features may give some social robots a commercial edge precisely because they make humans and machines less distinguishable. Even in an imagined Star Trek Holodeck future, it may still be possible to say “it's only a robot!” but increased realism nevertheless favors natural reactions.
Time domains
Depiction seems most relevant to longer time domains, and like for other media technologies when they were new, that domain is the current emphasis in robotics research. When people interpret, construe, imagine, or appreciate, this primarily involves “slow thinking” (Kahneman, Reference Kahneman2011), and C&F cite numerous good examples of how people reason about social robots in this time scale.
Media realism, however, causes quick responses that occur in seconds or less. “Fast thinking” research about social robots is relatively new but increasing. For example, when people touch a robot they show heightened arousal, similar to touching humans (Li, Ju, & Reeves, Reference Li, Ju and Reeves2017). And within seconds, people make judgments about the warmth and competence of social robots, just as they do for people in real life (Reeves, Hancock, & Liu, Reference Reeves, Hancock and Liu2020). Designers are focusing on other primitive features like eye contact (Kiilavuori, Sariola, Peltola, & Hietanen, Reference Kiilavuori, Sariola, Peltola and Hietanen2021), and how robots negotiate physical space (Hoffman & Ju, Reference Hoffman and Ju2014).
Discretionary framing
Media experiences are not only determined by media stimuli. People can choose a frame, at least temporarily. In one relevant research paradigm, people switch between interacting with media characters that they believe are either controlled by a computer or by another real person. The mere belief that people are interacting with a real person (and not with a character that only depicts someone real) results in greater arousal and better learning (Lim & Reeves, Reference Lim and Reeves2010; Okita, Bailenson, & Schwartz, Reference Okita, Bailenson and Schwartz2007).
It is also noteworthy that interventions designed to reduce the negative consequences of media often teach people, and particularly children, how primitive responses are triggered and how media professionals use them to control attention. This can reduce negative effects by teaching people to choose a symbolic experience rather than a natural one (Jeong, Cho, & Hwang, Reference Jeong, Cho and Hwang2012).
Sampling robots
There is likely far more variance between media characters (robots included) than variance between human responses to any one of them (Reeves, Yeykelis, & Cummings, Reference Reeves, Yeykelis and Cummings2016). Several years ago, we cataloged 342 social robots that were used in over 1,000 studies in the last decade (available at https://goo.gl/Gqpzkx). Variance between the robots is impressive and the catalog doesn't even include some of the most interesting current products (e.g., there are no experimental studies that use sex robots as stimulus material; Döring, Mohseni, & Walter, Reference Döring, Mohseni and Walter2020)
The point of the catalog is to show that any selection of a single or few robots can be easily biased. Consequently, stimulus sampling (Clark, Reference Clark1973; Cummings & Reeves, Reference Cummings, Reeves, Jussim, Krosnick and Stevens2022; Judd, Westfall, & Kenny, Reference Judd, Westfall and Kenny2012; Yarkoni, Reference Yarkoni2022) is critical for social robots. A discussion based on robots that are dinosaurs, sex companions, soccer competitors, or dance partners, will be different than one based on characters from Michelangelo and Shakespeare and the most common (and least exciting?) robots in health care and children's learning.
Conclusion
Symbolic and natural experiences are both important for understanding how media characters are experienced. One is not more important than the other, and one cannot be explained by the other. The switching that occurs between these frames offers a different answer to the author's social artifact puzzle about how people can think media characters are real, and at the same time realize that they are mere mechanical artifacts. They are both. And what matters most is how either experience works, which is applied when, and how they might interact over time.
Financial support
The author reports no funding related to this commentary
Competing interest
None.