Peer Review

What’s Peer Review?

If you’ve ever attended an intro session at your school’s library, you’ve listened to the librarians lower their voices and speak in reverential tones about peer review: the touchstone that separates rigorous research from mere opinion, hearsay, and the untutored opinions of your know-it-all roommate. At its most basic level, peer review is a form of quality control, a process that scholars came up with in the 1700s to protect the public from shady information. It gets its name from the fact that before a study is published, it’s evaluated by experts in the field—“peers”—who decide whether the work passes muster.

Here’s how the process is supposed to work, at least in theory. A researcher designs a study, gathers data, writes up results, and sends it off to a journal. The editor scans the study to make sure it fits the publication’s scope and then forwards it to reviewers. The reviewers assess the article’s merits: Have the authors interpreted their data correctly? Do the claims fit the evidence? Were the procedures used the correct ones? Are the conclusions justified? If the reviewers deem that the paper is flawed beyond repair, it receives a flat-out rejection. When Sam was starting out, he had the dubious honor of being rejected by some of psychology’s most prestigious journals.

If, on the other hand, a paper shines so brilliantly that there’s not a single thing to fix, it merits a straight acceptance (Sam’s still waiting). What tends to happen a lot of the time is that an article is placed in a nebulous category, known as “revise and resubmit,” that wavers between “reject” and “accept.” In this all-encompassing catch basin, the editor summarizes the reviewers’ criticisms and offers the authors a chance to fix their paper and give it another go. However, the opportunity to resubmit carries no guarantee that the resubmission will be published. If the revised manuscript is sent to new reviewers—something that can and does happen—the manuscript can languish in revise-and-resubmit-land for ages.

Even if a study is competent and avoids fatal errors, it can still be rejected if the reviewers conclude that it says nothing new or rehashes points already in the scholarly literature. Conversely, reviewers will sometimes accept a paper that’s not ready for prime time if they think it might spur researchers to take up new questions and open new lines of inquiry. If you think the process of peer review sounds complicated, laborious, time-consuming, and pulling-your-hair-out frustrating for anxious authors, you’re right. Case in point: a 2022 study that Sam’s team did on teaching students the techniques in this book took a year and a half and three different revisions before it finally appeared in print2 — Figure unavailable in the source files.