Dear Player: The Sensing Sim
Catacombs of Solaris opposes the two-dimensionality of the screen with the three-dimensionality of rendered space in a surprising and dramatic meditation on the limits of knowledge.
The following is a transcript of a talk I gave at British DIGRA 2018:
This abstract proposes we look more deeply at how mechanics in games can articulate novel kinds of sense-perception at the edges of ludic and narrative experiences, and in forms distinct from the previous scholarship more concerned with the experiential implications of virtual reality (Helsel, Meckler 1991 et al). I’m interested in games which wrestle with Haraway’s cyborg and Bogost’s Alien Phenomenology seriously and conceptually in a world where we all too often see sensory augmentation as instrumental progress. Indeed, I think we often see virtual environments as places which fulfil or extend our senses, when they also have a strength in questioning them.
With the rise of biohacking, Augmented Reality and wearables/hearables/implants, the human sensorium is being reimagined and reconstructed by technologies. In AAA games, the dream of expanding sensation often takes the form of utilitarian UI design which extends the affordances of vision in order to uncritically empower the player (See ‘Horizon: Zero Dawn’ (2017), ‘Deus Ex: Mankind Divided’ (2016), Assassin’s Creed’s eagle vision et al). However, game design can be more critical and inventive when dealing with sensory augmentation, and has the capacity to allow us to performatively reflect upon issues in phenomenology.
In ‘Unfinished Swan’ (2012) and ‘Scanner Sombre’ (2017) we are presented with alternatives to both player-empowering ‘second sights’ and the reductive and potentially ableist representations of sensory impairment as a limit to play. By analyzing how these first-person games can re-frame and interrogate sense-perception by stressing experiential difference in ‘supernatural’ environments, I identify shared and conflicting aesthetics in these case studies as a starting point for future research into what I term ‘Sensing Sims.’ In these examples the player can perform kinds of ‘alien phenomenology’ (Bogost, 2012) which avoids reductive or superheroic accounts of sensory impairment or enhancement and instead explore difference, alternative sensations, and the perspectives of the non-human.
‘Unfinished Swan’ and ‘Scanner Sombre’ enact thought experiments which deeply involve players affectively, visually and kinaesthetically, placing them in ‘blank’ or indirectly visible three-dimensional environments, and posing the mapping and navigation of these spaces as the central mechanics. In Unfinished Swan’s white world, hurled ink coats surfaces in black. By moving and inking the environment the player triangulates space. In Scanner Sombre’s cave system, coloured point data sprayed onto the black environment leaves a spectrum of colour indicating depth, but also slowing and elaborating sight in time by balancing fidelity and effort in painting the environment.
By using fictive prostheses (ink and scanner), they propose alternatives to both mainstream first-person privileging or enhancement of sight through UI, and to the conventional and normatively framed mode of engaging with exploration through lighting and colour. The implications of Thomas Nagel’s epistemological thought experiment in ‘What is it like to be a Bat?’ (Nagel, 1974) are here explored and expanded upon: can we process alien sensations through metaphor?; can we performatively change our sensory relationship to the world through play?
In both my examples the first-person protagonist reflects on epistemic issues concerning spaces they can only sense at a remove by throwing paint or LIDAR point-data onto their environment, casting surfaces into relief. Central to both is the problem of visual and tactile ‘noise’ and the ambiguity of inside and outside. Like echo-location, the space of play is initially blank/mute, but with an excess of player activity the environment becomes too cacophonous, and in playing with the shades in-between the player must wrestle with the limits of their own senses as they triangulate space.
Unfinished Swan grapples with visual closure
In Unifinished Swan the player must compare the surfaces of objects relative to themselves in a manner analogous to echo-location. The act of ‘looking’ has to take place in motion in order to reveal the third dimension or distinguish a relief sculpture from a sculpture in the round. However, interacting too much with the environment causes it to lose form like over-worked clay, converting it fully from white to black makes three-dimensions read as two. In Scanner Sombre our LIDAR scanner has different settings like photoshop brushes, from spraying localised areas with different densities of dots, to quickly gesturing out the space with a 360 radial scan.
Interestingly, and even more explicitly than in Swan, these different spray patterns convey the importance and necessity of embodied sensations – there is no seeing without a contingent, specific and dynamic perspective. While stationary, the regimented dots of a radial scan super-impose the fleeting impression of a sphere in any space, until we move and the optical illusion collapses. However, unlike Swan, which imagines the player to be a sighted individual in conventional space, Sombre has the player wearing a fictive VR headset such that line of sight does not matter to the player – we see all points from every other point, the ghost of a level, giving us no means of distinguishing between the near and far side of objects. In the mist of point data, reverse and obverse become indistinguishable. Merleau-Ponty’s principle of the distinction of figure and ground is played with here: Are we inside a cave or outside rock?
In Scanner Sombre the player's handle on space and matter is always partial and contingent
In both Swan and Sombre, there is also an interesting flattening of ontology (Bogost). Our understanding of objects mutates into an appreciation of Things – subjects and objects in new configurations and hierarchies. Combinations and elisions between everything in proximity prompt new relationships, new units and new affordances. Through a scanner sombre the table and chair next to it can be one contiguous being, while between the splashes of light and ink in a hidden environment our minds race to project possible structures and fantasies of correlation.
Video Games/Audio Games
The examples I've been outlining play on the interesting mismatch of tight folly work and de-realised visuals, but it is worth bearing in mind that there exists a whole movement dedicated to audio games in opposition (and complement) to 'video games.'
Lastly, and most intriguingly of all, these examples propose interesting relationships between time and perception. In Unfinished Swan we inhabit the metaphor of a blank canvas. By projecting ink around the environment we participate in it’s becoming – but, as the title reminds us, this space is unfinished: we are unable to tell whether the environment exists (hidden) before our attempt to sense it, or whether the act of sensing of ‘inking’, is what gives it form. Even more mind-bendingly, Scanner Sombre reifies fleeting traces at explicitly fixed points in the past – the geometry of a level could change behind us, but visually we would never know until re-mapping it. At multiple junctures this is played on for affective horror – the silhouettes of figures appear and disappear like the Angels from Doctor who, and fluid materials like water are impossible to see except by the absence of fixed point data. We are given, I suggest a vegetal kind of perception, like a plant growing around the contours of the environment towards an imagined light at the end of a cave, building up accreted sensations of the objects we touch through time.
“If we take seriously the idea that all objects recede interminably into themselves, then human perception becomes just one among many ways that objects might relate. To put things at the center of a new metaphysics also requires us to admit that they do not exist just for us.”
The Deleuzian study of affective intensities and modes of being in Colin Cremin (2015) and Alexander Galloway (2012) has recently highlighted the multi-sensory aspects and affective technological interfaces at work in games, and I am thankful for the growing body of work on aesthetics in games from likes of Graeme Kirkpatrick, Melanie Swalwell, John Sharp et al. What I find interesting in my case-studies, however, is their particularity and their philosophical implications. I also want to suggest that there is much to be gained from exploring phenomenological and epistemological questions articulated by games which lie on the edges of the ludic and the narratological. Scanner Sombre and Unfinished swan give us spaces to explore and reflect on phenomenology, without suggest transparent or unmediated translations of ‘alien’ sensations.
In these ‘sensing sims’, I argue, the player watches themselves in the act of watching in a manner which brings various sensations into relief, while preserving something of the ineffable.
And so my questions to you are: Can we meaningfully change our sensory relationship to the world through play? Does the ‘sensing sim’ I’ve outlined represent an interesting or significant development of the art game or walking sim (one beyond narrative?)? Is there a space for a queering of the senses here, and/or does my discussion of sensory difference rely on problematic ableist assumptions?
Merlin Seller
BIBLIOGRAPHY
Aarseth, Espen. (1997) Cybertext: Perspectives on Ergodic Literature. Baltimore and London: Johns Hopkins University Press.
Bogost, I. (2012) Alien phenomenology, or, What it's like to be a Thing. Minneapolis: University of Minnesota Press. (Posthumanities, 20).
Bolter, J. D., & Grusin, R. A. (1999). Remediation: Understanding New Media. Cambridge, Mass: MIT Press, 25.
Cremin, C. (2015) Exploring videogames with Deleuze and Guattari. Abingdon-on-Thames, UK: Routledge Ltd.
Deleuze, Gilles, and Félix Guattari. (2004) A Thousand Plateaus: Capitalism and Schizophrenia. London: Continuum.
Galloway, A.R. (2012) The interface effect. Cambridge: Polity.
Grodal, Torben. (2003) ’Stories for Eye, Ear, and Muscles: Video Games, Media, and Embodied Experiences’, in The Video Game Theory Reader, Mark J.P. Wolf and Bernard Perron, eds., New York and London: Routledge.
Helsel, Sandra K., and Judith Paris Roth. (1991), Virtual Reality: Theory, Practice, and Promise. Westport: Meckler.
Massumi, Brian. (2002) Parables for the Virtual: Movement, Affect, Sensation. Durham & London: Duke University Press.
Merleau-Ponty, M. and Landes, D.A. (tr.) (2012) Phenomenology of perception. London: Routledge.
Nagel, Thomas. “What Is It Like to Be a Bat?” The Philosophical Review, vol. 83, no. 4, 1974, pp. 435–450.
Giant Sparrow. (2012) Unfinished Swan. Sony Interactive Entertainment
Introversion Software. (2017) Scanner Sombre. Introversion Software.