My response to the NIH RFI on preprints The NIH released a request for information on preprints: https://grants.nih.gov/grants/guide/notice-files/NOT-OD-17-006.html. My responses are below. Types of interim research products you or your organization create/and or host. I blog and perform research at Omnes Res: http://www.omnesres.com/. I share my work on my own site, through public forums such as BioStars: https://www.biostars.org/p/179077/, youtube: http://bit.ly/2g1kxS9, or GitHub: https://github.com/OmnesRes. Although I personally don't view blog posts any differently than peer-reviewed articles, blog posts are not indexed by Google Scholar and are seen by many as simply "brainstorming" and not actual literature. Blog posts also have the issue of not being under version control or having a permanent archive. As a result, I write up my most important work with LaTeX and submit it to preprint servers. I would rather spend my time working on interesting projects than dealing with journal after journal to share my work, and preprints allow me to do this. In addition, without an institutional affiliation or grant support the cost of publishing would be an unnecessary financial burden for me and it is free to post articles on preprint servers. Feedback on what are considered to be interim research products, and how they are used in your field. I believe anything that is potentially useful, clearly presented, and easily accessible should be considered an interim research product. Currently the majority of scientific research is never shared, and therefore most grant money essentially gets wasted. For example, a sequencing dataset will never get posted to the Sequence Read Archive if it doesn't lead to a publication and will instead sit on a hard drive somewhere. If a protein structure doesn't lead to a publication it will never get deposited in the Protein Data Bank. I understand it can take time to upload to a public database, but if researchers had more incentive to share datasets, protocols, code, etc. I'm sure they would make the time. Insight on how particular types of interim research products might impact the advancement of science. I've had nothing but positive experiences with preprints. For example, as soon as http://www.oncolnc.org/ was finished I posted a preprint. OncoLnc currently gets 50K hits per month despite not being indexed by PubMed and never being presented at a scientific conference. I constantly receive emails from users thanking me for making OncoLnc and how it has helped them with their work. Yes, I could have shared OncoLnc without a preprint, but I assume most of my users learn of OncoLnc through word of mouth and without an accompanying preprint I don't know if users would have taken OncoLnc as seriously and shared it with their colleagues. I also have a story of how preprints can accelerate science. I learned of the GRIM test through a preprint, https://peerj.com/preprints/2064/, and immediately made a web tool for the authors. I also extended the GRIM test and posted a preprint of my own, https://peerj.com/preprints/2400/, all before the GRIM test was published in a peer reviewed journal. Feedback on potential citation standards. I don't see any reason why interim research products should not be cited. As long the interim research product is posted somewhere with clear version control and archiving standards I would be comfortable citing it in my work. Some researchers are concerned about having citations split between a preprint and the peer-reviewed article, but Google Scholar currently sums the citations together. Insight on the possible need and potential impact of citing interim products on peer review of NIH applications. Whether we like or not, most scientists are not altruistic and need incentives to behave as such. Currently the system incentivizes researchers to keep their work secret until it appears in a peer-reviewed journal. Researchers also have no incentive to make unpublished data available. Allowing more work to be listed on applications will serve to promote a culture of data sharing and rapid dissemination of results. Advice on how NIH reviewers might evaluate citations of interim research products in applications. Research products should always be evaluated on an individual basis, but I also believe it is acceptable to use the quality of previous work as a proxy. If an applicant has a clear history of high quality formal research products, then there is no reason to believe that the interim research products would be of any less quality. Alternatively, if a researcher has a history of publishing questionable research then the interim products will need to be highly scrutinized. Any other relevant information. Upon learning of preprints I found it frustrating that they were not indexed by PubMed. Yes, many preprint servers are indexed by Google Scholar, but so much is indexed by Google Scholar that it is easy for articles to slip through the cracks. In addition, researchers enjoy showing others how many articles they've written, and I'm sure many PubMed queries are simply authors typing in their own names. As a result, I decided to make a preprint search engine with similar search capabilities to PubMed, PrePubMed: http://www.prepubmed.org/. And I eat my own dog food. I use PrePubMed to find research and some of the useful articles that I've found I'm not sure sure I would have found if I only used Google Scholar, and obviously would not have found if I used PubMed. I believe preprints are in general of higher quality than published research because researchers go out of their way to post preprints. Why would you go out of your way to post work you're not proud of? In contrast, work in PubMed is often only published because researchers need the publication for their resume or for a grant application. I am not the only one to recognize the need for a preprint search engine. There is also search.bioPreprint: http://www.hsls.pitt.edu/resources/preprint and OSF preprints: https://osf.io/preprints/. In addition, ASAPbio has proposed the creation of a Central Service that would allow for searching all preprints. I believe there is a need and desire to have preprints indexed by PubMed. Perhaps they could be kept separate from MEDLINE articles, but the option to search for them should be available nonetheless.
A site by Jordan Anaya