This message was sent to ##Email##
|
|
|
|
|
| |
| Can Algorithms Replace Editors in the Peer Review Process?
|
    |
Marcie Granahan,
NFAIS Executive Director
|
The drive toward open science has prompted a re-imaging of current peer review processes. One of the more disruptive alternatives is the open peer review system created by F1000, in which the submitted paper is first published, referees are then invited to review the paper, the paper is then revised as necessary, and finally indexed based upon successful approval of the referees [see full article here]. The F1000 system has been adopted by the Wellcome Trust and Bill and Melinda Gate Foundation as a requirement of their funded grants [see presentation here]. The goal is to bring more transparency and efficiency to the peer review process.
However, two new studies may challenge several of the assumptions upon which the F1000 system is based.
According to a recent study in PS: Political Science and Politics, peer review that utilizes a unilateral editor decision system — where papers were read by reviewers but the editor maintains final say about which papers are ultimately published — resulted in fewer low-quality papers published compared to other systems examined [see full article here]. The study reveals that active editors — who execute independent decisions using reviewer comments instead of merely following the up-or-down vote of reviewers — produced the highest quality papers.
When it comes to article quality, a skilled and experienced editor appears to win out over algorithms that select articles for publication based on the ratings of qualified reviewers. The study also found that subfield-specific journals tend to publish more high-quality papers than general-interest journals.
Subfield-specific journals also tended to attract more reviewers than general-interest journals, according to a recent study in Scientometrics [see full article here]. The study found that the degree of difficulty in finding reviewers predicted manuscript scores, and that recruitment difficulty was the best predictor for article rejection.
Possibly a more organic process, in which reviewing work becomes more integrated into research, and reviewers self-select the submissions s/he wants to review rather than waiting to be invited could aid in improving the quality of the published articles. However, finding willing academics to review manuscripts is becoming increasingly more difficult for a number of journals, and there is no guarantee self-selection wouldn’t result in a disproportionate distribution of reviewers.
Whatever future path peer review takes, it will continue to be a group effort, based on an altruistic community that is motivated by the greater good.
|
Past insights and reflections on industry developments written by NFAIS Executive Director Marcie Granahan are now archived in NFAIS Community Forum. Feel free to post your comments about these and other key topics there. |
A brand new way to discover scholarly content. Learn more here. |
LSE Impact Blog
Since SocArXiv launched last year, researchers have uploaded almost 1,500 papers. With expanding outreach and community building, the system is heading toward greater growth and impact. Some papers we host are works in progress, others are under review or already accepted at traditional journals, some are shared as free versions of paywalled papers already published, and many include replication packages of data, code and other research materials.
READ MORE
Inside Higher Ed
Is it peer reviewed? A yes generally means that some experts, somewhere, think a given piece of research represents the best in its field. But a new study of peer-review processes within political science found that the majority of accepted papers will be evaluated by the average reader as below a journal’s publication standards.
READ MORE
The Scholarly Kitchen
Finding willing academics to review a manuscript is getting more difficult for some journals. Unfortunately, this difficulty may also be negatively affecting the judgment of journal editors, a new study reports. The paper, “Difficulty of recruiting reviewers predicts review scores and editorial decisions at six journals of ecology and evolution”, was published in the October issue of Scientometrics. Its author, Charles Fox is a professor of evolutionary and behavioral ecology at the University of Kentucky and the current editor of Functional Ecology.
READ MORE
NFAIS
The Center for Scientific Review at the National Institutes of Health (NIH) has signed an agreement with Elsevier, the information analytics company specializing in science and health, to support improving the peer review process of NIH grant applications by using Expert Lookup, Elsevier's tool that identifies scientific experts.
READ MORE
Nature
Millions of articles might soon disappear from ResearchGate, the world’s largest scholarly social network. Recently, five publishers said they had formed a coalition that would start ordering ResearchGate to remove research articles from its site because they breach publishers' copyright. A spokesperson for the group said that up to 7 million papers could be affected, and that a first batch of take-down notices, for around 100,000 articles, would be sent out “imminently”.
READ MORE
Inside Higher Ed
ResearchGate, a popular tool used by scholars to share their work, is taking down many researchers' work, apparently in response to demands from publishers.
Recently a group of scholarly publishers signaled that they had had enough of ResearchGate.
READ MORE
LSE Impact Blog
Authors seek to publish the results of their research in the best journal possible, one that most closely fits their topic and is likely to have the biggest impact. But of the 34,000+ active scholarly peer-reviewed journals, how do authors go about choosing a specific one to submit to? Many studies have examined this submission decision process and the complex collection of competing criteria.
READ MORE
Digital Doughnut
According to the latest predictions, digital transformation is coming of age and becoming a part of major corporate strategies. AI has taken a step ahead and its algorithms are being set to allow machines to learn by reading data sets – without any special programming.
READ MORE
The Cointelegraph
Social platforms have exploded in popularity over the past decade and are now a cornerstone in our daily interactions. People around the globe use these networks to share information with each other and mutually create value. That’s the theory, anyway. Many social networks are plagued by trolls, fake news that spreads like wildfire, and antisocial interactions between users.
READ MORE
The Pitt News
Benjamin Ogrodnik said he is often frustrated when traditional publication methods he uses are unable to include films and project methodology as part of the final product.
“A finished article is just a really refined argument. The story is in the creation of it,” Ogrodnik said. “There is so much I have to leave out, like conversations, memories and interviews.”
READ MORE
Wired
Imagine you're a scientist. You published a paper in a reputable journal a couple of years ago, and that paper has made an impact in your field of study. It has shaped your own research and has inspired others to pursue similar lines of investigation. Thanks to that paper, you're enjoying a certain status among your peers.
READ MORE
Research Information
Paul Kudlow says the publishing industry is facing a monumental challenge: How do we ensure research is read?
Up to half of the 8,000-plus articles published daily are only read by their authors and journal editors. An effective, scalable method to maximise the reach and discoverability of published content is needed.
READ MORE
The University of Kansas
Reading 10,000 poems to search for certain themes would be a daunting task, no matter how passionate the reader is about the topic. But the evolution of digital technology has not only made that possible, it’s making it possible to change the way people think about poetry. That’s what Whitney Sperrazza hopes to do: analyze poetry in a new way and take digital humanities to a much wider audience at the same time.
READ MORE
Times Colonist
Unlike the works of William Shakespeare or the mathematical proofs of Albert Einstein, the newest works in literature and science are now usually written and stored on a computer.
But whether kept on discs, computer hard drives or old-style magnetic tape, digital works come with preservation and storage issues that are only just being realized.
Lisa Goddard, the librarian responsible for digital scholarship at the University of Victoria, says many of the issues stem from simple lack of experience. We have yet to grasp the how or even why of digital preservation.
READ MORE
Forbes
Over the past few weeks my inbox has seen a steady stream of emails on the topic of big data and AI research ethics. From the NSF-funded PERVADE study to industry initiatives to roundtables and discussions at the Association of Internet Researchers annual meeting in Tartu, Estonia recently, it would seem data ethics has hit its stride. Yet, should we have hope that there will actually be change, or is it just too late for big data ethics?
READ MORE
Phys.org
Like other sciences, medical research is generating increasingly large data sets as doctors track health trends, the spread of diseases, genetic causes of illness and the like. Effectively using this data for efforts ranging from stopping the spread of deadly viruses to creating precision medicine treatments for individuals will be greatly accelerated by the secure sharing of the data, while also protecting individual privacy.
READ MORE
Enago Academy
The quality of research often improves with good collaborators. In our modern world, answering complex questions requires a team of experts. The Open Science Prize (OSP) was set up to promote international research collaborations. The OSP rewarded biomedical academic research and encouraged crowdsourcing of open data to help researchers make breakthroughs.
READ MORE
NFAIS
Former Macmillan President, Senior Advisor at Providence Equity, and Director at Education and Media Companies to Join Preeminent Research and Learning Company.
John Wiley and Sons, Inc., a global research and learning company and one of the world’s leading publishers, recently announced that Brian A. Napack has been named its new President and CEO, effective December 4. Mr. Napack succeeds Matthew Kissner, who has served as interim CEO since May of 2017. Mr. Kissner, a 14-year veteran of Wiley’s Board of Directors, will remain as Chairman.
READ MORE
Scientific Computing World
Elsevier has announced it is donating its Unified Data Model (UDM) to The Pistoia Alliance, a global, not-for-profit alliance that works to lower barriers to innovation in life sciences research.
The UDM is an XML file format originally developed by Elsevier to improve the upload of external data sets into its tools. It will now be developed and extended under the stewardship of The Pistoia Alliance, with the ultimate aim of publishing an open and freely available format for the storage and exchange of drug discovery data.
READ MORE
NFAIS
MyScienceWorks-A Global Platform for Researchers, Institutions, and Publishers
Tuesday, November 7, 2017 2:00 PM EST
Managing Digital Objects in an Expanding Science Ecosystem
Full-Day Workshop, November 15, 2017
Lunch & Learn: Are Your Communicating for Consent? Preparing for the EU GDPR
Single Presenter Event November 28, 2017
NFAIS 2018 Annual Conference
Information Transformation: Open. Global. Collaborative
Wednesday, Feb 28, 2018 1 p.m. - Friday, Mar. 2, 2018 2 p.m.
The NFAIS Career Center is the premier one-stop place for employers and job-seekers in the information services field to make the right connections. Click here to view all job opportunities or to post an open position.
|
|
|
|
7701 Las Colinas Ridge, Ste. 800, Irving, TX 75063
|
