We’ve been writing a lot about peer review recently, including a journalistic primer on peer review and a look at single-blind peer review and other forms peer review has taken to fight potential bias in a review. We know finding and easily accessing peer-reviewed content is really important to our Article Galaxy customers. Peer review feels so critical to scientific publishing - as a signifier of quality and as a signifier of reliability - that we assume it’s always been around. So, has it?
Yes...but not exactly as we think of it now.
In this post we’ll look back at the concept of “peer review” in the very earliest days of journal publishing. It turns out that early formative approaches to journal selectivity, peer review, and reliability are curiously topical 350 years later.
This blog post was inspired and informed by research done by Dr. Aileen Fyfe and her team at St. Andrews University in Scotland, who have been writing “The Secret History of the Scientific Journal.” It’s fascinating stuff!
Peer Review by the Peerage
Henry Oldenburg is credited with inventing the scholarly journal in 1665 in London UK. As the first secretary of the recently formed Royal Society, Oldenburg edited and published the first issue of the first science journal, Philosophical Transactions. Although he published Philosophical Transactions as a private commercial venture, his operation was break-even at best.
As editor, Oldenburg used his wide network of correspondence and his affiliation with the Royal Society to attract submissions--which ranged from letters of “natural philosophy” (now better known as “science”) observations and theories to summaries of talks given at the Royal Society headquarters in London, news and book reviews. Oldenburg made all the editorial decisions in selecting the content, often selecting articles to stimulate interest and discussion around a particular topic.
And here’s where the “peer review” comes in.
Oldenburg himself was a natural philosopher, so one could argue that he was a peer of other early scientists writing for his journal. But more accurately, the “peer review” came in the form of the imprimatur of the Royal Society. In the 17th Century in the UK, all publications needed a royal approval or license for publication. The Royal Society was a licensing body and the president of the Royal Society reviewed each issue of Philosophical Transactions. The head of the Royal Society was a “peer” as a fellow natural philosopher, but also in that he was a member of the peerage, or aristocracy. The review - given at the issue level, not an article level - was not focused on the quality of the science. Rather, the objective was to make sure the content did not include sensitive topics, like blasphemy, treason or sedition.
This kind of filtering out of inappropriate content, while not making value assessments of remaining content, is now taking place at bioRxiv (the life sciences preprint server). While not really filtering for blasphemy or sedition, Cold Spring Harbor Laboratory (the sponsor of bioRxiv) check to make sure that submissions do not contain non-scientific content such as pornography or creationism.
Peer Review for Scope, but Not Validity
By the 1770s, the Royal Society had formally taken over publication of Philosophical Transactions. It established a committee to handle editorial responsibilities (aka what to publish), which reviewed the content not so much to determine the validity of the work, but to establish the boundaries of good science.
According to Fyfe, the committee would not allow publication of things like supernatural claims of “women giving birth to rabbits or children vomiting worms,” but wanted to focus on observations from the natural world. In fact, the early Royal Society imprimatur from this period and into the early 20th century included a statement that explicitly indicated the veracity of the content was the sole responsibility of the authors, not the Royal Society. So when “peers” or committees of researchers were reviewing content, they were making directional decisions about topics to discuss, but were not endorsing the validity of the claims.
This practice of selecting appropriate content without assessing the impact of that content lives on today. Mega journals like PLOS ONE and Scientific Reports select content based on the scope and fundamental quality of the research, but do not reject content for lack of novelty or perceived impact.
Collaborative Peer Review
French scholars of the Enlightenment were also publishing science by way of the French Académie royale des sciences. The French took the idea of editorial oversight two steps farther than their British competitors.
Manuscripts were sent serially to multiple individuals within the Académie, who would leave hand-written comments in the margin of the manuscript.This is the first occurrence of collaborative peer review, where reviewers openly share comments with subsequent manuscript readers.
Modern publications including eLife are now using collaborative peer review techniques to help the editor make publication decisions. Indeed, many publications are embracing collaboration during review in many interesting ways.
Reproducibility
The French reviewers or “rapporteurs” in the 1760s were specifically tasked with assessing the truth value of the claims within a particular paper. That is, not for all papers, but specifically for those submitted by non-members of the Académie. This is the first example of a real focus on reproducibility. The French reviewers attempted to recreate experiments and report on the success or failure. Papers with a high level of reproducibility were published; papers with a low confidence level were not. This practice was eventually suspended, however, because it was deemed expensive and unsustainable.
There’s quite a bit of focus on reproducibility these days, with some even describing our current state of publishing a “reproducibility crisis.” Organizations such as Protocols.io have emerged to help publishers provide more detail than a methods section in a research article, which allows readers to more easily recreate experiments within their own labs.
More on Peer Review
To learn more about peer review history and how it resonates in the modern world, watch this video from the Association of Learned and Scholarly Publishers. It features Aileen Fyfe with other thought leaders discussing the past, present, and future of peer review in scientific publishing. It’s 90 minutes long but worth the time since it is well-produced and full of thoughtful discussion.
