Rodero, E. (2012). See it on a Radio Story. Sound Effects and Shots to Evoked Imagery and Attention on Audio Fiction. Communication Research, 39, 458-479. doi: 10.1177/0093650210386947.
Radio’s capacity to stimulate the creation of mental images in the mind of its listeners has long been acknowledged. Nevertheless, research into mental imagery has focused principally on the study of visual stimuli, whilst studies into radio itself have mostly concerned the field of advertising. In this study, we examine the influence of two stimuli associated with auditory processing on radio: sound effects and sound shots. The chosen context for the study is that of a fictional story, or audio drama, through which to measure the role of these stimuli both in creating mental images in the listener’s mind and in maintaining his/her attention. Our findings demonstrate that the inclusion of descriptive sound effects and especially of sound shots in a fictional radio drama increases mental imagery and that a relationship exists between this increase and the degree of listener attention.
Video (coming soon) Audio (Play below to listen)
Significance of the project
If there is one essential characteristic that can be attributed to radio which is common to practically all radio-related literature, it is its capacity to stimulate the creation of mental images in the listener’s mind. This characteristic is a constant in all the relevant literature appertaining to the radio as a medium of communication, although scarce empirical data is provided to back up this notion. However, despite this ubiquitous reference, compared with other fields, few studies relating to the capacity of radio to create mental images have been conducted to date. Indeed, while the process of creating mental images has long been the object of study from various perspectives, attention has focused mainly on visual stimuli rather than those which are audio-evoked.
Goals of the Project
The aim of the study was to ascertain the degree of incidence of two characteristic radio stimuli, sound effects and sound shots, in the creation of visual images and in listener attention when they are applied to a fictional audio story. This study is designed to advance in the impact of auditory structural features on human processing of audio messages, enhancing work in the field of mental imagery and attention. This is applied to a field little explored from the perspective of fictional radio stories, and by extension audio stories, and we particularly focus on two audio resources which are potentially visual: sound effects and sound shots.
Firstly, the study demonstrated that the story based solely on dialogues was the one that least stimulated listeners’ imagination and attention levels. This finding lends support to earlier studies in which the use of different resources in radio messages had yielded the same result (Bone and Ellen, 1992; Childers and Houston, 1984; Lutz and Lutz, 1977; Unnava and Burnkrant, 1991; Bolls, 2002), although these studies did not concern the specific genre of radio drama. This phenomenon can be explained due to the dual-code theory (DCT) and the propositional-representations theory (PRT). According to the former, the verbal nature of the stimulus would have triggered a mechanism more linked to semantic processing rather than visual/spatial processing, which would not benefit from double coding. In the case of the latter, the explanation is based on the activation of semantic structures in the working memory. As no perceptual structures (visual –sound effects– and spatial –sound shots–) are activated in response to a stimulus, the coding process in the memory, and by extension, the process for creating images in the mind would be hindered and delayed. Therefore, the listener has created fewer images, encountering greater difficulty and needing more time to do so. Likewise, the images were less intense, less well-defined, less clear and less lifelike. In addition to this effect, the stability of the verbal stimulus throughout the entire story, due to the lack of variation, has led to a reduction in the stability, interest and concentration when it comes to the listener paying attention to this story.
As far as sound effects are concerned, our study demonstrates that they are effective in stimulating images and in enhancing listener attention; the stories in which they are used, S2 and S4, have obtained a significant main effect for imagery and attention. This finding is consistent with the literature reviewed regarding applications in radio advertisements (Potter et al., 1997). Indeed, the studies carried out by Miller and Marks (1997) demonstrate that the creation of mental images generated by verbal messages is less intense and numerous than that generated directly by sound effects. However, no such investigation had been carried out into radio drama. Moreover, earlier studies had not involved the prior isolation of the function fulfilled by sound effects in a radio production, a factor which we consider relevant to the results of any investigation into the phenomenon.
As for sound shots, our study also demonstrates that their use in a fictional story increases both the creation of visual images and listener attention. The stories in which they are used, S3 and S4, have obtained a significant main effect for imagery and attention. These findings can be also interpreted in the context of Paivio’s DCT and Kieras’s PRT. The sound shots have activated auditory representations in the non-verbal spatial codes or propositional structures, which have heightened attention and the creation of visual images, though in this case in terms of space. According to the Theory of Spatial Representation (Kosslyn et al., 2006), establishing different distances between the characters has enabled spaces and actions to be more easily situated and, therefore, has provided the listener with a spatial representation (“where”), more lifelike. This increased sense of reality, triggered by the use of sound shots, has arisen because during auditory processing the listener hears events aside from merely acoustic signals: “the sounds of people and things moving, changing, beginning and ending, forever interdependent with the dynamics of the present moment. Auditory experience is always of a flow of sound, constant at times, rising or falling in intensity” (Forrester, 2002: 33). Consequently, in addition to characterizing this flow of sound, sound shots have organized the stimulus in perceptive terms by distinguishing between significant sounds (foreground) and secondary sounds (background), as in the figure-ground phenomenon. By simulating the structure of real space and capturing all possible relations between sound objects encoded in a coordinate scene, they gave the listener a greater sense of reality through which to better imagine the event represented in the scene. Indeed, an aspect of mental imagery relevant to McInnis and Price’s study (1987: 480) is its effect on the perceived likelihood of events: “the act of visualizing an event can make the event seem more likely”. Owing to this distinction, which has improved intelligibility, the listener has better been able to adopt a specific standpoint (point of listening), on the basis of which to create a subjective representation of the story, concentrating on actions performed by sound shots. As Beck (1998) points out, the listener, and thus his attention, is always positioned at the centre of the sound which determines the mixing and balancing of sound technically, in other words, his attention focuses on and is positioned at the foreground in the space, represented by sound shots (Chion, 1994). Accordingly, he establishes a subjective representation (Rodero, 2009: 243), which should lead more, highly intense images to be created in the mind, thereby encouraging the listener to keep and focus his attention in relation to a dialogue-based story. As a result, sound shots in this study, by stimulating auditory representations in the non-verbal spatial codes or propositional structures, containing and distributing the sounds in significant spatial frameworks, have heightened the listener’s attention, interest and concentration and have created a greater number of clearer, more intense, better defined and more lifelike images with greater ease and swiftness. Therefore, the findings of this study demonstrate that, in auditory processing also, spatial information becomes important when it comes to increasing the rate at which mental images are created and the listener pays attention.