Should more papers be retracted?
Most of the scientific literature is wrong, and yet it is allowed to continue to exist. Anyone who is an expert in a field can rattle off a long list of papers, even papers that are highly cited, that are complete bullshit and have set the field back and continue to do so. Most of these papers probably don’t contain fraud, or even outright errors, but rather poorly designed/performed experiments, misinterpreted results, questionable analyses, misuses of techniques/data, or contain results that are unreproducible due to artifacts or small sample sizes.
And since these reside in the published literature with the stamp of peer review who knows who is reading them and how their results are being used. Maybe someone is planning a clinical trial based on one of these papers. Maybe a government is making policy decisions based on one of these papers. You just don’t know.
During my time in medical school I actually happened to experience a consequence of one of these papers. The paper was “Genome-wide non-mendelian inheritance of extra-genomic information in Arabidopsis”. I was aware of the paper because we had a good laugh about it in my college genetics class. Most of the results in the paper have been explained by outcrossing, and the first author who is still an active researcher has only published one paper since 2005. So color me surprised when in my medical school lecture on epigenetics by the popular and respected genomics researcher Patrick Grant I saw an RNA backup genome listed as a known mechanism and this paper was cited. I attempted to explain to Dr. Grant on the medical school forum that this paper likely wasn’t evidence of a backup genome, but to no avail.
How many classes of 150–160 medical students at UVA sat through a lecture where a backup RNA genome was taught? Is it taught at other schools? Sure, anyone who has read a high school biology book would know that RNA isn’t stable enough to be a long term store of genetic information, so this is a poor example, but who knows how many papers that are ridiculed in their field are being taught as fact by professors across the country or used as a basis for research projects.
So why aren’t papers which the majority of the scientific community thinks are wrong or misleading not retracted? It’s because it would make the journal look bad, and it is the journal’s decision whether or not to retract a paper. Umm, conflict of interest anyone?
I don’t believe journals should have the power to retract papers, or accept papers for that matter, or really any power at all. I believe an organization of scientists should be in charge of maintaining the accuracy of the scientific literature. How about the National Academy of Sciences? What the fuck do they do besides publish garbage in PNAS? On Reddit when a comment gets enough downvotes it gets buried further down in the list and eventually is hidden unless you specifically click on it. Science publishing will likely never be as advanced as Reddit, but could we at least get large warning signs on questionable articles? Sure there are posted corrections, but those are mainly for small errors. Yes, you can comment on articles, but almost no one does this.
Luckily some people have taken matters into their own hands. For example PubPeer allows people to anonymously comment on articles, but these comments are mainly limited to errors or fraud. What we need are experts in a field commenting on the current reputation of a paper. Maybe once a paper has received a certain number of citations, and therefore is being highly read and influencing a field, it undergoes another round of peer review. This peer review would be more extensive than the original review process, and may include repeating analyses or even experiments. If the paper fails this round of peer review a statement of concern would then appear on all versions of the article. But if the paper passes this round of review it would receive a gold star or some actually meaningful stamp of approval.
If you would like to comment on this post it is available at Medium