Mimesis and Interactivity

Here comes a bunch of blogging! Fasten your seat belts.

================

So yes, we got a Kinect at our house. I am the very model of the modern gamer tech geek. As an incremental change to the wand-driven interface design of the Wii and PS3, I admire it. I’m far more fascinated by the really imaginative hacking of the powerful capabilities of the device, and the unintended ends to which they may lead. I confess I was also a bit disappointed that the interface didn’t function like a combination of “Minority Report” and the Bat-Computer to the extent that I’d secretly hoped it might.

What frustrates me most about the Kinect, however, is not the device itself but the common misapprehension of some middlebrow game and digital media critics, most prominently Seth Schiesel of the New York Times, that the Kinect is the future of a naturalistic, real-world mode of interacting with digital appliances and media. Schiesel states the hope succinctly: that the banishment of game controllers, iPod dials, keyboards and other control devices in favor of intuitive motions of physical bodies and natural language commands is the end of a geek-favoring barrier to the consumption of digital media and the use of digital tools and the beginning of a great democratization of the digital.

This is in the end a very geek-oriented way of imagining why some media practices seem to cohere to geeks, that design is destiny, that technology intrinsically favors or excludes users because of its particular material or conceptual nature, usually a feature or architecture that a critic or designer believes can be and should be changed.

I don’t entirely disagree with this perspective. Design matters, and it matters in ways that are not purely a mirror of sociology or culture. This is even true of the Kinect or Wii or Sony Move control systems in particular. Schiesel and others are perfectly correct to say that kicking a virtual soccer ball or doing a virtual exercise routine with a motion-capture system is intuitive in a way that using a multi-button controller is not, and that this intuitive design permits many people to play some digital games when they would otherwise think that the effort of learning a control scheme doesn’t justify the reward of playing the game.

What bugs me about the middlebrow celebration of the downfall of the multibutton controller and its kindred devices (keyboards, etc.) is the naive understanding of mimesis buried inside that enthusiasm. The driving faith here is that representation and lived experience should have a 1:1 correspondence in order to rid ourselves of the work and difficulty that comes from a slippage between the two. There’s at least a kissing-cousin resemblance between this view and older positivist ideas, lingering on in some scientific and social-scientific circles, that we should tinker ceaselessly with language until all ambiguity is banished from it and it thus can be used for the efficient description of the real world.

Let’s say that Microsoft continues to hammer the bugs and quirks out of the Kinect, making its recognition of both language and motion closer and closer to how we hear and interpret speech and action with our own perceptual systems. Let’s even pretend that there won’t be a more and more obvious uncanny valley of some kind as it does so. As the system becomes more and more mimetic, at least in theory, will that truly rid of us of complex control schemes that only a geek could love?

Of course not, at least not if digital games work with the unreal, the imaginary, the impossible. What an odd thing that anyone should wish for games to become more restrictively mimetic to “reality” at a moment when digital technologies are otherwise opening up representational possibilities in film and television.

Stick for a moment even to sports games. A programmer could make a better and better Kinect-controlled soccer game, but if that is only going to involve those actual physical routines we use in a real game of soccer (which are themselves not something that human beings are born knowing, and are in some cases anything but intuitive: a game where you can’t use your hands? Not exactly a natural idea for a primate with opposable thumbs), two problems will quickly arise. First, if the action I see on the screen is to be synchronized with the action I perform in real-world space, the action can be in general be no faster or slower than my real physical motion. Maybe you’re different than me, but I don’t play soccer at the speed of a Ronaldo or Beckham. So nothing in the digital game can appear to be enhanced from the world unless everything is enhanced or exaggerated to the same degree, and every computer-controlled player or physical action has to be as slow and boring to watch as I am in real life. Second, I can’t do anything that doesn’t involve a match between an on-screen avatar’s motions and my motions. The avatar can be first-person or third-person, but I can’t do something like control multiple avatars, or control markedly non-human objects or creatures unless I learn to do something very imaginative, abstract or counter-intuitive with my body in real-world physical space. I can be a tiger in a Kinect game, but that either has to involve translating my normal bipedal ape motions into the motions of a four-legged feline or it has to involve my mimicking the motions of a four-legged feline.

From there, it’s a pretty short step to the Kinect version of having to memorize a series of finishing moves. It’s not as if this is something that digital media force upon our normally naturalistic, intuitive bodies. A boss fight in World of Warcraft has always seemed to me to have a very strong analogy to choreography, and I can easily see a Kinect-style future for a game of that kind where getting the right sequence of heals on the tank would look more like T’ai Chi than keyboard typing. But all of that will involve something as intricate and complex as contemporary controller interfaces (or real-world multiperson dance recitals). Without that slippage-filled interfacing complexity, I won’t be able to be a Jedi in a Kinect game: a game could interpret my raised hand as a Force choke, a push as a Force push (much as it looks in films and cartoons) but I can’t tell my avatar to do a eight-foot tall Jedi backflip without a gesture which is very fundamentally not an eight-foot tall backflip.

We can’t be freed of the work of representation, the ambiguity of language. Why should we want to be? That is like imagining a freedom from life itself. It will be all to the good if the Kinect makes a game designer deep in his or her cubicled warrens wonder if the best way to connect a player’s actions with the attack of a fantasy warrior in an imaginary world is X O O X left trigger long-hold-on-X as opposed to the player making a fist in the air and waving it around. Anything that unsettles byzantine practices of culture by reminding us of their contingency is good, because that’s what catalyzes the creative discovery of the novel and unfamiliar. That creativity will be stillborn if it has to satisfy the expectation of the Schiesels of the world that they will never again have to learn something unfamiliar in order to control the unfolding of the imaginary.