The Limits to Shill

I continue to feel pretty diffident about the controversy among anthropologists about the Human Terrain Team and other uses of qualitative social science by the U.S. military over the past decade or so. The issue for me is not whether this is an intrinsic misuse of anthropological or qualitative research, blurring the lines between legitimate fieldwork and other uses of ethnographic methodology. That’s partly because I feel that anthropologists have a tendency to draw overly stark lines between their own disciplinary traditions and all other forms of fieldwork, usually in the process implying that all other kinds and styles of qualitative fieldwork are both ethically and methodologically suspect. Ethnography is spying of a kind. Or to flip it around, there’s plenty of kinds of intelligence-gathering that take place in plain sight.

The issue for me is not whether institutions like the police or the military can legitimately use anthropology. It’s whether those institutions are prepared to accept what they’d learn by doing that kind of research honestly, or whether the use of anthropologists or other researchers is just a way to put sugar on a brute-force shit sandwich. At least in some cases, what our military or other militaries might have learned in the past by doing genuine, sophisticated anthropological research is that there is no way to achieve the declared objectives of a military mission, and that you either alter the objectives or end the mission. In Afghanistan, for example, I’m not sure that it would lead to successful counter-insurgency if all ISAF Forces became profoundly culturally sensitive. It would help, as would the casual overuse of aerial bombardment or brutal intrusion into communities. But culturally sensitive or not, the ISAF don’t live there in the truly long-term sense, they’re not a part of villages where the residents have to make choices about whether to collaborate or tolerate Taliban forces who have an intimate knowledge of the social networks in that village and work with people who do live there.

So it’s fine to study something using any method or specialists you like. Just be ready to hear what that method is going to tell you.


A very different example of the same issue. The New York Times today has a piece on how market researchers and others are using computer analysis to try and identify embedded sentiments within online discourse. You don’t need a computer to do that, just use the hermeneutical engine located inside your own head and do some reading.

Use computers if you like to spot trends, I suppose, but to really know what a surge in expressed feelings means in any given Internet forum means, you really have to be a long-term follower of the flow of conversation and communication at that forum or others like it. Saying you’ll do “sentiment analysis” with computers on online forums is like saying you’ll do “sentiment analysis” in a literary work. I don’t doubt that something roughly along those lines is possible, but to really make any sense of it is going to take a human being doing interpretation the old, slow and human way. An algorithm that spots a change in word usage in a long-running forum is only alerting its minders to an event: making sense of that event is another matter.

The issue is not at all that market researchers are trying to analyze sentiment at any rate, or even the methods that they’re choosing to analyze it with. It’s whether they’re prepared to understand what they find out, and to act upon it. Just to take an example from the article, when Wal-Mart looks at “Labor Force and Unions”, it finds that there is a lot of negative sentiment. The Times suggests a public-relations strategy as a response to that finding. Wrong answer. If you’re really doing sentiment analysis, you’d understand that this strategy is a great way to make more negative sentiment. Because the reason there is negative sentiment is that Wal-Mart’s labor policies are pretty bad.

That’s like suggesting that you send a shill poster into a fan forum to promote a bad new movie or video game, to counter negative sentiment you’ve detected. If your shills are new posters, everyone almost immediately detects the shill and all you’ve done is increase the negative sentiment. If it’s an old, established poster, you’d have to pay that person off a great deal to get them to shill, unless they routinely shill for things, at which point see, “increase the negative sentiment” above. A non-shill poster who starts sounding like a shill is obvious; a non-shill poster who somehow manages to promote something in an appropriately cynical, somewhat denigrating way is not really helping to counter negative sentiment in the first place.

What you really learn by doing sentiment analysis (by computer or the better human-brain-reading-things way) is, “Don’t do whatever the hell it is that is pissing people off if you really can’t afford to piss them off”. If you’re Wal-Mart and you’re seeing some impact from your bad labor-management, do better by your employees. If you’re sitting on top of a $100 million cinematic turd, try to avoid shitting out celluloid crap in the future. If you can’t or won’t change, don’t think that there’s a magic trick buried in “sentiment analysis”.

What happens when there’s a new raft of consultants selling some new thing like sentiment analysis is that as the new specialized service develops, the people selling it are increasingly pressured by clients to make the information come out in a way that is soothing and tranquilizing, that either says that the problem doesn’t exist or that there’s some superficial cosmetic strategy that lets the organization go on doing something flawed or self-interested while somehow getting rid of all the criticism and negativity that behavior has incited. The most avid buyers of that kind of information tend to be the mid-rank bureaucrats and managers who are less concerned with long-term missions and more concerned with their own short-term prospects, and they can dance a beautiful duet with consultants or pet researchers who are willing to whisper them sweet nothings as long as the salaries or payments keep on coming.

For institutions that really want to know something that they don’t already know, and think it’s important for the long-term mission, then there has to be some willingness and preparation to hear information that’s not palatable and to act on it in that form, up to and including ending the practices or projects involved outright. Because the issue is really not the use of novel methods: it’s what’s true in the world around you, and about what keeps some organizations from being able to hear or understand those truths. The world itself doesn’t conform to what middle-managers and bureaucrats sometimes need it to be. Sometimes it really is murky, or ambivalent, or confusing. Sometimes it’s exactly the opposite to the way that the CEOs and generals and political leaders pretend it is. Sometimes if you want to know how people feel, you’ll just have to join them down in the fogginess of daily life and grope your way around the hermeneutical mist like everyone else.

This entry was posted in Information Technology and Information Literacy. Bookmark the permalink.

5 Responses to The Limits to Shill

  1. Doug says:

    “The vast space of the Borders [between Poland and Russia] is a well-known fact, which every general must first accept then ignore; a strategist who treated the expanse of the Borders with due respect would never fight at all.” – Norman Davies in White Eagle, Red Star

    Though of course top-level decisions are often taken for reasons separate from the conditions in a particular area.

  2. evangoer says:

    Wal-Mart’s labor policies are bad, Wal-Mart knows that its labor policies are bad, and Wal-Mart has no intention of fixing its labor policies.

    But this has nothing to do with sentiment analysis… or focus groups, polls, usability testing, eye-tracking studies, A/B bucket testing, or necromancy. Is there something specific about sentiment analysis that you think is much, much worse than any of these other tools?

  3. Timothy Burke says:

    No, the point is precisely what you say: Wal-Mart has no intention of fixing those policies. So sentiment analysis is just another of those tools that institutions buy to tell them something they already know, or tell them something that they don’t want to know in a form that they’d prefer to know it in. E.g., in some cases, it’s just another form of opening up sheep to peer at entrails so that you can deliver some performance art that pleases the powerful.

  4. nord says:

    Keep saying WalMarts labor policies are bad and enough people will link the two phrases. Walmart has a few tools to change the terms of debate and eventually sentiment … one is anecdote. Stories are much more powerful than statistics regarding labor law violations. A few local stories of single women moving up from checkout lines to regional management are remembered. The outcome of the large class action lawsuit against Walmart is not. The second is shifting the terms of the debate … “live better” is the current slogan, but there are other versions of the same message. Walmart isn’t above buying off the environmentalists either – you’ll be seeing a lot more solar cells on the roof and windmills in the parking lot of your local Walmart in the future.

    But holding aside their response, we all know that the Walmart and labor conditions are all about the UFCW protecting their turf as walmart is taking a lot of market share from their unionized stores, right? I have not seen evidence that Walmart is worse than other non-unionized employers in terms of labor practices – they are just more effective at killing off union companies … The UAW has been trying to create that same sentiment with the non-unionized transplants. I can’t tell whether they have failed because too many elites would rather drive a prius than a buick or because the public can see their naked self-interest…

  5. Valerie says:

    You write:

    What you really learn by doing sentiment analysis (by computer or the better human-brain-reading-things way) is, ??Don??t do whatever the hell it is that is pissing people off if you really can??t afford to piss them off??.

    Amen. This is why promotion, the passing out of free samples of a new product is so risky. If your product isn’t good, people will find that out FASTER if you give it away.

    Years ago, my sis did advertising for McDonald’s. For a time, McDonald’s biscuit product was being left in the dust by Hardee’s biscuit product. So the ad firm did the research and found that the issue was pretty much that the Hardee’s product tasted better in taste tests. There was not a damned thing advertising could do to change the perception of people who had tasted both and decided they preferred the other product. The McDonald’s franchisees were NOT happy with this finding. Weirdly, instead of pressuring McD’s to come up with a better recipe, they wanted the ad firm to come up with a better campaign.

    Truth is powerful stuff. But only if you pay attention to it.

Comments are closed.