Kenny Chow (Hong Kong) & D. Fox Harrell (MIT)
Saturday, May 19th
Buchanan A201
Introduction
In today’s digital environments, we not only watch, but also interact with animated graphics in user interfaces, videogames, and computer-based artworks. The experience of interactive animated images relies on both sensory and motor apparatuses, constituting an embodied cognitive process. [1] We call this process “material-based imagination.” Through sensorimotor experience of interactive animation, a user recalls and infers associated visceral sensations. Furthermore, users elaborate imaginative experiences from the embodied sensations based on dynamically generated animated visuals. Drawing upon theories of conceptual blending [2-4], material anchors [5], and neuroscience results regarding mirror neurons [6], we present our concept of “elastic anchors” to explicate the meaning-making process in interactive animation. We analyze examples from a range of digital artifacts, including graphical user interfaces, viral websites, videogames, and computer-based artworks.
Theoretical
Idea In this paper, we build on Gilles Fauconnier and Mark Turner’s theory of conceptual blends and Edwin Hutchins’s theory of material anchors. In particular, we argue that interactive animated images function as what we call “elastic” anchors for conceptual blends. Elastic anchors not only “hold” conceptual structures in place like material anchors, but also “embody” dynamic sensational meanings in sensorimotor experience for users to elaborate imaginatively. We argue that material anchors, whose physical structures faithfully and stably represent conceptual elements for reducing memory and cognitive processing loads in goal-specific human-performed computational tasks, may not necessarily apply to process-driven imagination-laden activities such as storyboarding in filmmaking or practicing Chinese calligraphy.
The material artifacts being produced in many expressive activities blend with mental concepts in the creators’ minds, forming new imaginative images as input into the next iteration of the material artifacts. In animation, this reflective and iterative process becomes instantaneous and continuous. Consider for example, shadow puppeteers adjusting their hand movements in real-time to create shadow characters. Such phenomena mobilize a motor-sensory feedback loop in which imaginative meaning construction and elaboration take place through successive conceptual blends of animated images and mental images, converging toward interpretations of the images as being “lifelike.” Interactive animation, as enabled by today’s real-time control and rendering technologies, trigger this kind of reflective process that we call material-based imagination.
Illustrative
Analyses User experience of elastic anchors is ubiquitous in digital environments. Here, we present analyses of how users of digital animation elaborate imaginative meaning at the immediate and the metaphorical levels. For example, see Figure 1 (a phone showing animated water level), 2, and 3 below for a condensed sample analysis.
Implications
Through these analyses, we reveal the important role of imaginative blends in making multiple levels of meaning through sensorimotor experience of interactive animated visuals. The analyses collectively support the need for an embodied cognition approach to understanding animation phenomena and, more broadly, to expressive production in cognitive science research.
(See the attached PDF for figures and references)
