Responding to Timothy’s comment: I certainly agree that moving development efforts “away from ‘projects which are too big to fail’ … and towards lots of little, robust, consciously experimental projects” — this pretty clearly enables more rapid learning and evolution if the projects contribute to the “DNA” of subsequent projects in ways that appropriate reflect their successes and failures.

I guess I’d claim that sometimes projects do have to be big to work. This issue shouldn’t become a roadblock but an approach that can’t handle it ultimately will need to be changed / augmented, and that presents a danger of sliding back into a “big project” mode…

Continuing with Timothy’s comment: we should prefer projects “which require little or no continuous staffing and supervision by NGOs but which have to set concrete goals or make concrete predictions about their impact, which are held accountable for what happens as a result of the project.” Actually, I think this is not exactly wrong, but skew to what is actually needed, in a way that will perpetuate some serious problems.

Projects can be viewed as having two important results: their immediate success or failure on their goals, and their contributions to future projects in terms of understanding “how to get it done” and also sometimes “what to get done.” Focusing on the “concrete goals” has virtues but also makes it hard or impossible to learn important lessons from projects. Even more important, it mis-directs the focus of the larger scale institutions away from the learning process in favor of easily measured outcomes which may not be anywhere near the most important results.

To sharpen this point, let’s consider a project to distribute mosquito nets which uses the currently most favored “consensus” approach, and clearly fails. The project could be a huge “success” in terms of the future of the overall effort if it turns out it was executed well and failed for reasons more or less intrinsic to the consensus approach.

A less dramatic and more realistic example is a project that tries the consensus approach but finds they need to modify it significantly to make it work. In the end they are neither particularly successful or unsuccessful, but they do invent / discover / refine some tactics that are helpful, and also identify some that don’t work in their situation. This doesn’t fit the clean popperian falsificationist model, but of course is much closer to how researchers learn from real experiments.

Results like these, if studied and respected, will motivate and guide improvement of the consensus or replacement with something better. But if the projects are simply evaluated in terms of success in achieving goals, we are throwing away a lot of opportunities to learn from analysis of the fine structure of the results. The “accountability” trope basically tends to imply “failure makes projects losers to be scorned, success makes projects heros to be emulated” which to me bears an uncomfortable resemblance to the “personal responsiblity” meme that seems pretty pernicious but also culturally pervasive.

Being able to learn effectively on this large institutional scale requires a very rich and well maintained context of reporting, analysis of success and failure, propagation of new perspectives, maintenance of appropriate norms of discussion and epistemic rectitude, etc. That kind of context doesn’t just happen, it needs to be cultivated and sometimes enforced. We have a variety of institutional examples where it is done better or worse. Unfortunately in this kind of example if done worse (as I suspect it usually is in development) it will kill substantial numbers of people and possibly wreck entire economies and regions (as e.g. the post-soviet union economic interventions wrecked that region).

If I was running the QA department of a large development organization (and probably no such department exists, which is part of the point) I’d try to define my role as monitoring and improving the epistemic culture.

I’m not sure why the institutional culture of development projects is so much less effective than e.g. the institutional culture of airplane safety. Maybe it is just that the constituents for good development are so much weaker relative to the institutions. But this is what I’d like to fix.

Why would we think that governments and ngos are any more easily persuadable by straightforward goal-directed rational argumentation than localities x, y and z? Is it just that they need to be better educated?

Essentially it seems to me that development organizations should look at the way cell phones have disseminated (to great developmental effect) across rural Africa and ask, “So how can we by design do something comparable to that? Small changes that make big differences?”

I pretty strongly agree with Scott and you (and I think Packard, I couldn’t quite figure out his position from your account) about the nature of the problem. Scott is an anarchist so may not feel the state, or state supported NGOs have a positive role. But I take it you and Packard feel it does — as do I. Even Scott probably sees a role for very large scale collective action.

So let’s take things in that direction. Given these systematic and well documented failings for large organizations, what should be their positive social role? What culture, goals, aspirations, status etc. should we get them to adopt? “Rational plans”, “getting people to follow expert advice” etc. are clearly the wrong answers. But what are the right answers?

Getting people to reject the current modes of thought because they are flawed usually doesn’t work. Displacing current modes with better ones sometimes does.

To be less high minded, and more self-absorbed, I think the same rules of action at a distance might apply to the implementation of my department’s assessment plan.