Guest Post: Scholarly Mad Libs and Peer-less Reviews

Health Care Renewal presents a guest post by Marjorie Lazoff, MD, a Board certified internist with a clinical background in academic emergency medicine.  She is currently a full time freelance editor and independent consultant specializing in evidence-based clinical content and medical informatics.

On December 17, 2014, Scientific American published an investigative report by journalist Charles Seife documenting a new and curious form of scholarly publication fraud, For Sale: “Your Name Here” in a Prestigious Science Journal. As an editor and supporter of evidence-based medicine I am both appalled by, and sympathetic to, how such widespread fraud could take place unnoticed.

Seife describes how he discovered the doctored writings:

The dubious papers aren't easy to spot. Taken individually each research article seems legitimate. But in an investigation by Scientific American that analyzed the language used in more than 100 scientific articles we found evidence of some worrisome patterns—signs of what appears to be an attempt to game the peer-review system on an industrial scale…

…This is not a simple case of plagiarism. Many seemingly independent research teams have been plagiarizing the same passage. An article in PLOS ONE may eventually lead to 'our better, comprehensive understanding' of the association between mutations in the XRCC1 gene and thyroid cancer risk. Another in the International Journal of Cancer (published by Wiley) might eventually lead to 'our better, comprehensive understanding' of the association between mutations in the XPA gene and cancer risk—and so on. Sometimes there are minor variations in the wording but in more than a dozen articles we found almost identical language with different genes and diseases seemingly plunked into the paragraph, like an esoteric version of Mad Libs, the parlor game in which participants fill in missing words in a passage.

Another example virtually eliminates the likelihood of coincidence:

There is no such thing as a 'Beggers funnel plot'…the proliferation of 'Begger's' tests [were discovered] by accident. While looking for trends in medical journal articles, papers [were found] that had almost identical titles, similar choices in graphics and the same quirky errors, such as 'Begger's funnel plot.'

Seife’s investigative reporting revealed that China was the source of most of his “fill-in-the-blanks” research. Further,

Much of the funding for these suspect papers comes from the Chinese government. Of the first 100 papers identified by Scientific American [and listed at the close of his article], 24 had received funding from the National Natural Science Foundation of China (NSFC), a governmental funding agency roughly equivalent to the U.S.'s National Science Foundation. Another 17 acknowledged grants from other government sources.

Seife suspects that most research probably began as legitimate work without intent to deceive, but somewhere an author or service was added to help ensure publication through the necessarily arduous manuscript review process.

The culprit?

A quick Internet search uncovers outfits that offer to arrange, for a fee, authorship of papers to be published in peer-reviewed outlets. They seem to cater to researchers looking for a quick and dirty way of getting a publication in a prestigious international scientific journal.

Seife’s investigation goes undercover, 60 Minutes style:

In November Scientific American asked a Chinese-speaking reporter to contact MedChina, which offers dozens of scientific 'topics for sale' and scientific journal 'article transfer' agreements. Posing as a person shopping for a scientific authorship, the reporter spoke with a MedChina representative who explained that the papers were already more or less accepted to peer-reviewed journals; apparently, all that was needed was a little editing and revising. The price depends, in part, on the impact factor of the target journal and whether the paper is experimental or meta-analytic. In this case, the MedChina rep offered authorship of a meta-analysis linking a protein to papillary thyroid cancer slated to be published in a journal with an impact factor of 3.353. The cost: 93,000 RMB—about $15,000.

Finally, the corrosive effect of this particular fraud on scientific and medical publication is real:

Publishers at the moment are fighting an uphill battle. 'Without insider information it's very difficult to police this,' Clinical Endocrinology's Bevan says. CE and its publisher, Wiley, are trying to close loopholes in the editorial process to flag suspicious late changes in authorship and other irregularities. 'You have to accept that people are submitting things in good faith and honesty,' Bevan says.

That is the essential threat. Now that a number of companies have figured out how to make money off of scientific misconduct, that presumption of honesty is in danger of becoming an anachronism.

Were this the only threat currently facing research journals today! Last month, Retraction Watch published an article describing a known and partially-related problem: fake peer reviews, in this case involving 50 BioMed Central papers. In the above-described article, Seife referred to this BioMed Central discovery; he was able to examine 6 of these titles and found that all were from Chinese authors, and shared style and subject matter to other “paper mill-written” meta-analyses.

Retraction Watch agrees:

It would seem that a third party, perhaps marketing services helping authors have papers accepted, was involved.

Problems with peer review are longstanding editorial fodder. For a description of another recent peer review scam, this one involving authors hijacking researchers’ identities, see the article also written by Retraction Watch editors and published last month in Nature.

On Friday, in response to requests by several publishers, The Committee on Publication Ethics (COPE) posted a statement on inappropriate manipulation of peer review processes 

While there are a number of well-established reputable agencies offering manuscript-preparation services to authors, investigations at several journals suggests that some agencies are selling services, ranging from authorship of pre-written manuscripts to providing fabricated contact details for peer reviewers during the submission process and then supplying reviews from these fabricated addresses. Some of these peer reviewer accounts have the names of seemingly real researchers but with email addresses that differ from those from their institutions or associated with their previous publications, others appear to be completely fictitious.

COPE recommends, among other things, the retraction of articles based solely on fraudulent reviews. Retraction Watch’s announcement earlier today of a MacArthur Foundation grant to help fund a comprehensive and freely available database of retractions could not have come at a better time!

Seife and Retraction Watch have documented new forms of published research fraud among third world researchers. Certainly the solution is not for editors and readers to suspect all papers from specific countries; there are ample instances of research fraud emanating from English-speaking researchers and top U.S. institutions. Research from around the world is critically important, particularly although not exclusively in the basic sciences, emerging infectious disease, and public health/epidemiology. Now that it has been identified, a common screening procedure for manuscripts at a journal can be adjusted to filter out this new form of plagiarism.

Sadly, it seems to me that fraudulent research of all types can flourish within a perfect storm of circumstances and factors: the globalization of science and medicine encourages non-or-limited English-speaking researchers to publish (or perish) in the highest impact English language journals; the proliferation of open-access wannabes, hybrids of every color and degree of sincerity, and other money-over-science journals and companies that rip off desperate and naïve researchers; a complicated, time-consuming and often author-unfriendly manuscript submission process; and journal editors who struggle with limited staffing and resources, necessarily arduous editorial processes, and the pressure of increasing numbers of worthy manuscripts deserving to reach the scientific and medical communities in near-real time. Research fraud is particularly destructive given traditional publishing’s ongoing struggle to survive the transformational Electronic Age; the pervasive if not perverse marketing of pharma, medical device companies, and self-promoting individuals and institutions using “unbiased” research; and today’s bizarrely anti-science culture.

Health Care Renewal is wonderful at calling out intentionally perpetrated health care events whose importance and implications can be debated, depending on one’s perspective and personal values. Here, I think, we have the reverse: there is near unanimity over the need to prevent fraudulent papers of any type from contaminating our research databases, as best as is humanly and technologically possible. There is also near unanimity among quality medical journals throughout the world, and internationally respected editor and publisher groups, to confront and solve these problems. The enemy identified by HCR is not always unrestrained greed or maliciousness. Sometimes, as in this case, the enemy is a cacophony of small circumstances and extraneous factors that could, if left unattended, invisibly erode something we all hold dear.

Without ongoing attention and support from the entire medical and science communities, we risk the progressive erosion of our essential, venerable research database, until it finally becomes too contaminated for even our most talented editors to heal.

Dr Marjorie Lazoff

ADDENDUM (30 December, 2014) - This post was reposted on the Naked Capitalism blog on 24 December, 2014.

ADDENDUM (7 January, 2015) - See also comments on DSHR's Blog.  

ADDENDUM (19 January, 2015) - This post was reposted by TruthOut on 10 January, 2015.