Jackson Doughart
articles   |   poetry   |   music   |   officiating   |   c.v.
Rethinking peer review

Ottawa Citizen, 09 August 2013


Imagine being at a dinner party where the conversation turns to the realm of ideas on any topic. Now suppose that each of your comments were very closely scrutinized, so much that before being able to say anything, more senior members of the group would have to know and approve of your comment in advance. Most people would find this to be an onerous constraint in the free exchange of ideas, yet it is the very way in which much of today’s academic discourse is conducted.

This is mostly thanks to the fetishization of peer review, which is believed to lend authority to academic publications. Here is how peer review works: when a scholar submits a paper to a recognized academic journal, that paper must be thoroughly vetted by more senior academics before it is even considered for publication. For some journals, the process can take up to a year, in order for enough “blind reviews” to take place.

The popular understanding of this procedure is that peer review is a kind of fact- and citation-checking process, designed to ensure that the author has not fabricated evidence or failed to observe the standards of scholarship. But peer review also involves vetting the argument being presented in the paper, ostensibly to ensure that the readers of the publication are not subject to a “bad” argument, as determined by the reviewers.

This added element is quite problematic, especially when one considers the existence of such lower human traits as pride and jealousy, from which academics are not exempt. Now, there are plenty of scholars who are not over-protective and who welcome challenges to their research, but many are not so humble, and are prepared to abuse the system for their own purposes.

Peer reviewing is supposed to be done by experts in the specific academic discipline within which a submitted article is written, and this qualification is predicated on the reviewer’s own contributions to the same field. This means that reviewers will often be vetting articles that pertain to their own work. Hence, there is many an opportunity to block perfectly legitimate arguments, especially when a scholar hopes to safeguard his own work from too much scrutiny.

One way that this is done is through the seniority convention associated with major journals. If one looks at the editorial board membership inside the cover of any leading publication, one sees the names of the most-recognized academics in the field. Being on that list effectively gives a scholar the right to review the papers that will constitute the journal’s upcoming issues. And if a paper is too critical of someone’s own work, the prospect of vetoing that piece will be tempting.

One example that I can share comes from a professor of mine, who submitted an article to a journal a few years ago which criticized an essay by another academic. He didn’t hear back for over a year and when he did, he was rejected. He later found out that the person doing the review of his article was the very scholar that his own piece was criticizing, who tried to block it so that his arguments would not be heard. So much for impartial review!

This strange and overrated process has a chilling effect on debate, which ought to be open to the widest possible range of views, especially in disciplines where ideas can be taken quite easily from the scholarly to the popular realm. But this problem is not endemic to the humanities, and contributes to the undeserved deference that “peer-reviewed science” enjoys outside the academy. As the late journalist Alexander Cockburn wrote:

“(O) ftentimes, peer review is nonsense. As anyone who has ever put his nose inside a university will know, peer review is usually a mode of excluding the unexpected, the unpredictable and the unrespectable, and forming a mutually back-scratching circle. The history of peer review and how it developed is not a pretty sight. Through the process of peer review, of certain papers being nodded through by experts and other papers being given a red cross, the controllers of the major scientific journals can include what they like and exclude what they don’t like. Peer review is frequently a way of controlling debate, even curtailing it. Many people who fall back on peer-reviewed science seem afraid to have out the intellectual argument.”

All of this invites the question of why the vetting process could not be streamlined, limited to the fact-checking duties that fall under the editor’s role in any medium. At present, there is an exaggerated premium placed on peer-reviewed publications in the academic vocation. To be considered for a position as a tenured professor, one’s curriculum vitae must include several articles bearing the adjacent symbol “R” for refereed. This encourages young scholars to tailor their own work to the prevailing attitudes and paradigms of the day, and discourages them from actually challenging much within their discipline. This is an unfortunate development for scholarship in general, which owes to itself and to the public a more open and free-flowing publishing process.





______________________________________
Jackson Doughart jdoughart (at) gmail (dot) com