Mimesis and Interactivity

Here comes a bunch of blogging! Fasten your seat belts.


So yes, we got a Kinect at our house. I am the very model of the modern gamer tech geek. As an incremental change to the wand-driven interface design of the Wii and PS3, I admire it. I’m far more fascinated by the really imaginative hacking of the powerful capabilities of the device, and the unintended ends to which they may lead. I confess I was also a bit disappointed that the interface didn’t function like a combination of “Minority Report” and the Bat-Computer to the extent that I’d secretly hoped it might.

What frustrates me most about the Kinect, however, is not the device itself but the common misapprehension of some middlebrow game and digital media critics, most prominently Seth Schiesel of the New York Times, that the Kinect is the future of a naturalistic, real-world mode of interacting with digital appliances and media. Schiesel states the hope succinctly: that the banishment of game controllers, iPod dials, keyboards and other control devices in favor of intuitive motions of physical bodies and natural language commands is the end of a geek-favoring barrier to the consumption of digital media and the use of digital tools and the beginning of a great democratization of the digital.

This is in the end a very geek-oriented way of imagining why some media practices seem to cohere to geeks, that design is destiny, that technology intrinsically favors or excludes users because of its particular material or conceptual nature, usually a feature or architecture that a critic or designer believes can be and should be changed.

I don’t entirely disagree with this perspective. Design matters, and it matters in ways that are not purely a mirror of sociology or culture. This is even true of the Kinect or Wii or Sony Move control systems in particular. Schiesel and others are perfectly correct to say that kicking a virtual soccer ball or doing a virtual exercise routine with a motion-capture system is intuitive in a way that using a multi-button controller is not, and that this intuitive design permits many people to play some digital games when they would otherwise think that the effort of learning a control scheme doesn’t justify the reward of playing the game.

What bugs me about the middlebrow celebration of the downfall of the multibutton controller and its kindred devices (keyboards, etc.) is the naive understanding of mimesis buried inside that enthusiasm. The driving faith here is that representation and lived experience should have a 1:1 correspondence in order to rid ourselves of the work and difficulty that comes from a slippage between the two. There’s at least a kissing-cousin resemblance between this view and older positivist ideas, lingering on in some scientific and social-scientific circles, that we should tinker ceaselessly with language until all ambiguity is banished from it and it thus can be used for the efficient description of the real world.

Let’s say that Microsoft continues to hammer the bugs and quirks out of the Kinect, making its recognition of both language and motion closer and closer to how we hear and interpret speech and action with our own perceptual systems. Let’s even pretend that there won’t be a more and more obvious uncanny valley of some kind as it does so. As the system becomes more and more mimetic, at least in theory, will that truly rid of us of complex control schemes that only a geek could love?

Of course not, at least not if digital games work with the unreal, the imaginary, the impossible. What an odd thing that anyone should wish for games to become more restrictively mimetic to “reality” at a moment when digital technologies are otherwise opening up representational possibilities in film and television.

Stick for a moment even to sports games. A programmer could make a better and better Kinect-controlled soccer game, but if that is only going to involve those actual physical routines we use in a real game of soccer (which are themselves not something that human beings are born knowing, and are in some cases anything but intuitive: a game where you can’t use your hands? Not exactly a natural idea for a primate with opposable thumbs), two problems will quickly arise. First, if the action I see on the screen is to be synchronized with the action I perform in real-world space, the action can be in general be no faster or slower than my real physical motion. Maybe you’re different than me, but I don’t play soccer at the speed of a Ronaldo or Beckham. So nothing in the digital game can appear to be enhanced from the world unless everything is enhanced or exaggerated to the same degree, and every computer-controlled player or physical action has to be as slow and boring to watch as I am in real life. Second, I can’t do anything that doesn’t involve a match between an on-screen avatar’s motions and my motions. The avatar can be first-person or third-person, but I can’t do something like control multiple avatars, or control markedly non-human objects or creatures unless I learn to do something very imaginative, abstract or counter-intuitive with my body in real-world physical space. I can be a tiger in a Kinect game, but that either has to involve translating my normal bipedal ape motions into the motions of a four-legged feline or it has to involve my mimicking the motions of a four-legged feline.

From there, it’s a pretty short step to the Kinect version of having to memorize a series of finishing moves. It’s not as if this is something that digital media force upon our normally naturalistic, intuitive bodies. A boss fight in World of Warcraft has always seemed to me to have a very strong analogy to choreography, and I can easily see a Kinect-style future for a game of that kind where getting the right sequence of heals on the tank would look more like T’ai Chi than keyboard typing. But all of that will involve something as intricate and complex as contemporary controller interfaces (or real-world multiperson dance recitals). Without that slippage-filled interfacing complexity, I won’t be able to be a Jedi in a Kinect game: a game could interpret my raised hand as a Force choke, a push as a Force push (much as it looks in films and cartoons) but I can’t tell my avatar to do a eight-foot tall Jedi backflip without a gesture which is very fundamentally not an eight-foot tall backflip.

We can’t be freed of the work of representation, the ambiguity of language. Why should we want to be? That is like imagining a freedom from life itself. It will be all to the good if the Kinect makes a game designer deep in his or her cubicled warrens wonder if the best way to connect a player’s actions with the attack of a fantasy warrior in an imaginary world is X O O X left trigger long-hold-on-X as opposed to the player making a fist in the air and waving it around. Anything that unsettles byzantine practices of culture by reminding us of their contingency is good, because that’s what catalyzes the creative discovery of the novel and unfamiliar. That creativity will be stillborn if it has to satisfy the expectation of the Schiesels of the world that they will never again have to learn something unfamiliar in order to control the unfolding of the imaginary.

This entry was posted in Games and Gaming, Popular Culture. Bookmark the permalink.

3 Responses to Mimesis and Interactivity

  1. jfruh says:

    I think one of the issues of game controllers is that you need to be able to somehow abstract away the controllers metally as you’re playing to really enjoy it. I am pretty klutzy and bad at quick responses, so when playing traditional video games with controllers I rarely get really immersed in the game. I don’t feel like I’m in a ninja battle with bad guys or whatever; I feel like I’m trying to push buttons as fast as I can, or in a specific sequence. It makes the whole thing not fun for me. Similarly, Rock Band to me always feels like I’m (badly) trying to push a series of buttons, not play guitar. I never get comfortable enough to let my imagination take over.

    On the other hand, I love turn-based strategy games like Civilization, because I have time to stop to think (and imagine) what I’m doing, even though that too just involves a series of keyboard presses. I guess I feel like I’m giving orders in a more abstract way; pressing control-b feels like giving a royal command, “build a city!” but pressing the A button doesn’t feel I’m telling my legs to jump.

    By contrast, Wii games (I haven’t played with the Kinect) make it easier for me to lose myself and see my game-avatar as me. I know my swings in Wii tennis aren’t exactly like a real tennis swing, but working the controller is less intrusive. I don’t know if this is a geek vs. non-geek thing (I’m pretty much most kinds of geeks you can name) as a comfort level with a certain type of input control, or a certain ability to semi-automate one’s fingers.

  2. Brutus says:

    Beyond the nerdgasm kewl factor of what’s now possible with various digital devices, you answered your own question with this:

    We can?? be freed of the work of representation, the ambiguity of language. Why should we want to be? That is like imagining a freedom from life itself.

    Freedom from life for a techno-Utopian or transhumanist is losing oneself within the objects of our own creation, immersed in virtual experience that resembles and replaces reality completely enough that to exist any other way is both redundant and undesirable because of all that, um, well, messiness, discomfort, and pain. If the human condition possesses a variety of negative attributes (which it does), removal to virtual life provides release from that suffering and its consequences — of a sort. When the interface becomes convincing enough, the product name should be Oblivion rather than Kinect or Wii.

  3. Bill McNeill says:

    The one word that leapt out at me from this post was “middlebrow”. When the New York Times employs a middlebrow critic whose obtuseness infuriates aficionados, you’ve truly arrived as an artform.

Comments are closed.