Unthinkable Things #12: The twilight of peer review?

A few years ago, it would have been unthinkable for serious academic journals to publish non-peer-reviewed work, or for universities to grant promotion on the basis of such work.

photo by David Michael Morris - some rights reserved (Creative Commons 2.0)

So says Dan Cohen, director of the Center for History and New Media at George Mason University, quoted in a recent article in the New York Times reviewing recent and forthcoming experiments by Shakespeare Quarterly, Postmedieval and repec.org. The flagship of the wiki world, Wikipedia, has had a mixed reputation at best in academic circles, where it's often mistrusted for its open nature and its lack of dependence on traditional academic structures to verify its contents. Wikipedia itself is not likely to become a source worthy to be cited in serious academic work - nor has it aspired to. But it has done a huge amount to promote and prove the principle that work in progress is likely to be improved through being exposed to the collective judgement of as broad as possible an interested audience, who choose to spend what Clay Shirky would call their "cognitive surplus" on engaging with it.

For the time being, the kinds of experiments described in the article are very much the exception rather than the rule. Whether they will in time eclipse the traditional peer review process - described by Katherine Rowe, an academic at the USA's Bryn Mawr College involved in the Shakespeare Quarterly trial, as "a settlement for a particular moment, not a perfect ideal" - will depend in part on the respective efficacy of the two approaches in generating good quality work.

But, as with so many aspects of change driven by digital technology, the speed, nature and extent of this change will also depend on the readiness of institutions - in this case, those who govern academic career structures - to accept it. Too fast, and there is the danger not simply that vested interests will be challenged but also that rigour will suffer. Too slow, and academia risks losing credibility as both pioneer and guarantor of the world's knowledge.