No, please. There are still fields where journals are both open-access and not pay-to-publish. This is because they are published by the same non-profit learned societies they always have been for many decades, and overhead costs are low.
Bring in a culture of paying peer reviewers, and suddenly authors (or the learned society) will be under more pressure to get more grant money to pay the new publication costs. Also, as someone who works in academia on an occasional basis, taking years off from funded positions at a time, I appreciate being able to write the occasional article on my own time and get it published without paying hundreds and hundreds of euro out of my own pocket -- fields with pay-to-publish traditions make it hard for non-permanent faculty to contribute.
I have an inside line to Elsevier. Elsevier will NEVER cut anything, bloat or otherwise.
They will, however, use their market dominance to punish people who rebel against their publishing machine. They've done it consistently every time there's the whiff of shakeup in the publishing industry, going back decades.
You are right, they will never do something considered good for the public or research, just because they want to retain their dominance and maximize profits.
For example, my old university needs to pay in order to access papers, which they sent in and paid to publish. It's completely crazy.
Fortunately there is https://arxiv.org/ and similar sites. And most researchers are happy to provide you their papers free of cost if you ask.
This is insightful, thanks, but there's a little problem here:
The cost is borne one way or the other, the $500 is a matter of comp, not cost.
If someone is going to review your work, that's an act of very material labour. That they are not really compensated for it, at all, is inconsistent with the basic premise of labour and comp.
You would not be a researcher if you weren't comped in some way, so the absence and/or misalignment of comp is the issue.
It's not free to publish and have 2 reviewers (!!) there is a very material cost that you're just not paying.
Allocating at $1K cost for publication more accurately reflects the labour involved in checking it out (at least a little bit).
It'd seem to me a lot of folks might want to take up the $500/review as a job to earn some reasonable income, they might even get good at it.
It's entirely plausible that one of the 'dysfunctions' of science is 'not enough review' and it's simply because there is no incentive structure for it at all.
We pay people for literally every other function in life. Doctors are paid to 'save lives' even though there is a moral obligation. The notion that we don't pay people for academic work seems odd.
Perhaps journals could identify the papers with the most promise and offer payment for review, or something like that.
Unpaid peer reviewing is certainly not incentive-free. For example, oftentimes one enjoys peer reviewing because the author cites some publications that are useful or important, but which one wasn't previously aware of. Sometimes the author makes some observation tangential to his/her main argument which proves an inspiration or key to one's own research.
These sort of things may be more common in the humanities versus hard STEM, but in my own field peer review is not generally seen as drudgery or exploitation.
As someone who is in a STEM field (CS) I would say that all the reasons you mentioned apply to me personally. There are some conferences where being a reviewer can be very tedious due to the massive number of submissions (many of which are unfortunately fairly poor quality) but I haven't had this problem in my own subfield.
Do you realize that the article authors are generally not paid?
> It's not free to publish and have 2 reviewers (!!) there is a very material cost that you're just not paying.
Who is the "you"? The author I think? So you do know that article authors don't get paid, you're talking as if they are the "customer" that should be paying?
Generally, for non-open-source journals, which are still the majority in most fields, it is subscribers, the readers, who are the customers who pay the costs (plus enough for a healthy profit for the publisher). (Usually university libraries. The same universities who are paying most of the authors, who don't get paid by the journals, but produce the content that makes a journal valuable).
Elsevier is quite profitable, with a high margin on revenue.
You are painting an economic model of publishing that I'm not sure is connected to the actual real world economics of scientific/academic publishing, it seems to be very hypothetical.
Depending on the country and the field, authors are often faculty who are presently being paid a meagre salary only to lecture, and researching and publishing is something they do on their own time unpaid.
Yes, paying researchers to 'review' could be a solution but incentives are still misaligned - there's far too much upside in publishing vis-a-vis reviewing. It's a prisoner's dilemma type thing and it would require a kind of broad, institutional practice to make sure everyone was helping out with the reviewing part.
Economic incentives are never perfect either, but they might help.
Since we have a giant oversupply of PhDs looking for positions it'd seem maybe this wold be nice work for them, and frankly anyone applying for any kind of job should have a big list of articles they've reviewed as one of those resume checkmarks.
Literally not at all what i'm saying, but I am apparently failing to communicate today, so I'll give up.
But yes, I think changes in "broad, institutional practice" are required to fix the current severe problems in academic publishing, of which a lack of peer review labor is only a small part.
"Is peer review also the assumption or implication of their role?
If not, should it be,"
This is what I was responding to assuming you were implying that in general should 'researchers also be paid to review, implied as part of their job description'.
As if to say 'reviewing should go hand in hand with publishing' as part of the normal paid activity of researchers.
That probably makes the most sense but would require a lot of institutions getting on board.
The system is wickedly broken. Is there a way to fix the reproduction issue in science that has been raging for decades, without affecting the good players? I'm not sure there is.
I've seen some journals do this. The problem TBH is that the amounts paid aren't enough to be an incentive. Typically £50-100. A good review takes a day to do (maybe more). And academics are probably paid £100/day or more anyway, and aren't very money-motivated.
I think a better approach would be publicity. Academics care about reputation a lot. If good reviewers were acknowledged by the journal, that would be a big incentive, something you could put on your CV.
Great point. Offering to pay a small amount could actually reduce the number of people willing to review, since they are currently motivated by a sense of duty. Reminds me of the Freakonomics chapter with the day care: https://freakonomics.com/2013/10/23/what-makes-people-do-wha...
Publons is a nice way of keeping track and I use it myself, but I think institutions are generally willing to take faculty at their word with respect to reviews we have completed. (Although my statement here is really just based on anecdotal evidence, so I could be wrong.
Journals should not shed out tiny payments for peer reviews by overworked faculty, they should hire teams of reviewers with diverse specialisations full time. By decent quality assurance on the massive amount of papers each year publishers could actually do something valuable for science.
I don't necessarily disagree with paying reviewers, but I think there is some value to having people who are actively working on problems in the same area being the ones doing the reviewing. Yes, it's additional work that we aren't paid for, but personally I enjoy it. I do agree though that for some it can create undue pressure so there may be something in the middle there that works well.
I understand this point, yet to correctly assess whether a typical paper in my field - for instance - can be trusted you would need a modern neural networks researcher, a statistician, a cognitive scientist, a neuroscientist and perhaps a physicist. All would have to know about common fallacies in their respective fields and should optimally discuss with another.
Faculty reviewers are not only overworked, they have accumulated biases, can be overconfident about their expertise and in the worst case have malicious incentive to prohibit publication of your results.
I’m certain that a team of specialists who review as their full time job would do a better job than at least half of the reviewers I encountered (especially those at high-profile AI conferences).
I enjoy reviewing too and am doing my best to be constructive, but I’m not sufficiently qualified for most of the offers I receive.
Would a person / team with the qualifications you mention best spend his time on reviewing such work, or should they also be doing some research.
My point is, the only way to get the necessary skillset for reviewing is to actually do research. Hence it is peer-review. Once you are doing scientific research, there generally isn't anyone left who is 'better at it then you' so the best you can do is have it be reviewed by people who are 'roughly as good at it as you'.
To ensure the long-term accuracy, we need both attempts at reproduction, and critical reading by people post publication. Both of those should have the option to cause the article to be retracted.
I suppose a lot of this is dependent on the field. I think anyone who has had their work peer reviewed has many stories of reviews from people who seem to have completely misunderstood the work in question. Certainly there can be accumulated biases and overconfidence among faculty reviewers, but I don't see how that would be less prevalent among paid reviewers.
That would be difficult in a fields with a vast range of specializations (physics for sure). They would really need an army of reviewers to cover everything from astronomy to particle physics.
Lots of these highly qualified researches are forced to leave science right now, the reasons all too often not being their qualification. There would be a large pool of good researchers to draw from for this.
Can each individual journal (say Nature Neuroscience, Neuron, EMBO, Acta, Lancet, Trends, Annals of Neurology, PLoS, and Current Biology) each feed the dozens of neuroscientist specialists who would be needed as employees to do these reviews?
How would each of those specialists retain currency in the field by just doing reviews?
Isn't that what the editor panels do more effectively today (of course farming out the detailed/specialist aspects to actual peer reviewers)?
Hiring: Are you aware of the discussion around Elseviers profit margins? Various fields are considering abandoning journals (or have abandoned them long ago, see cryptography) since it is increasingly difficult to identify the value of a for-profit publisher for science. They will need to redefine their business model if sufficient scientists stop playing along for free.
„Currency“: Broad expertise, knowledge about methods and ideas are gathered by reading literature in-depth, which most scientists have little time for in modern science (hence the reluctance to engage in a long peer review process). Working as a paid reviewer for a few years can lead to a scientist who is informed very well about the state of the broader field and its various fallacies, and could move on from there. We have very few of those.
Editors: We wouldn’t have various discussions around peer review if the current system would be working satisfactorily well and nothing needs to change.
Looking at the real-estate market and its assortment of minimal-value-add/maximum-value-extraction players (appraisals, title insurance firms, 75% of inspectors) as an analog, I suspect that directly paying for reviews will not likely improve the situation.
Perhaps a recognition by institution or university model would work, or softening the dollars (they are credits against APCs or something) might work.
Many universities take reviewing "into account " for the tenure-track process and performance reviews. but it counts really little compared to publications. Data sharing or management of ressources (databases, forums...) is completely ignored usually.
Like many other recent international hires at US universities, I'm on H-1B visa. If paying for reviews was the norm, I believe it would be illegal for me to review papers, even for free.
I'm not sure what to think about this. Peer review is an important way of contributing to the community, and every academic is expected to participate in it. Commercial publishers that make profits from free academic labor are parasites, and they should pay for the services they use. Yet immigration rules often make it difficult to earn any side income, so paying for reviews would exclude many (often less established) academics.
That is a good point and one I hadn't considered despite being an H-1B holder myself. I agree with your argument that paying for reviews under current immigration laws would be highly problematic for visa holders. It would either mean we would be expected to do it for free when everyone else gets paid, which would be unfair and potentially still not legal. Or we would be unable to review which I think is an important part of establishing oneself within the community.
I don't think the process would be less effective if papers were published without peer review, but were then questioned and challenged in public debate under editorial moderation.
Peer review isn't a single-goal process. It exists to introduce citations of existing work if they aren't already referenced, to question assumptions, and to find obvious errors. (Among others.)
All of these could be done in peer debate.
It's not as if the process is genuinely anonymous anyway. Of course it pretends to be, but it isn't.
Peer review dates from the days when information travelled as fast as a horse, and it made sense to include a review stage for basic quality control.
I'm not convinced it's a good model now, when information exchange is rather quicker.
The real problem is that journal publication is about career status, not about quality of work. Peer review is almost a side issue.
The scientific publishing industry is not just financially parasitic, it also introduces unnecessary delays in idea exchange. Public peer debate would eliminate some of that friction, but it would also have public consequences for reputation - and that might well be more effective for quality control than pseudo-anonymity.
At least in my field, I expect this would be a total disaster. One of the major advantages of peer review is that knowing the top tier publication venues, I can look to these to keep up to date on the latest work knowing that what I find is likely to be high quality. In the absence of peer-reviewed publication venues, I see two options:
1. Read everything I can find that is published and try to sort through it all myself to sort out what is good quality. I think in aggregate, this is likely to take more of researchers' time than contributing to the peer review process.
2. Focus only on publications from top quality institutions or researchers whose work I am already familiar with. This has the advantage of saving me time. But it also has the massive disadvantage that I am likely to miss quality work which comes from lesser known institutions or authors.
I don't think either of those two options is attractive. In my field, the delay of idea exchange is not really a problem. If people want to share their ideas, they can often publish a preprint in parallel with submission to a peer-reviewed journal. (Although some journals effectively prohibit this since it has the potential to break the anonymity of the review process.)
Yes, some of the purposes of peer review could be accomplished in the method of "peer debate" you mention. But I think such a process would be much less effective. Of course, that's not to say I don't think there are things broken about peer review that could use fixing :)
Can’t we jus my have a GitHub for science and get this over with? It seems silly to maintain this system that may have been functional when it started, but is now clearly behind the technology curve.
How would that address the core question being asked which is whether work spent reviewing should be paid? In any case, while there are problems with peer review, I don't think it's related to technology.
If the journal is paying a reviewer would there be more pressure in terms of turn around time? Or more pressure to accept contentious papers because the journal expects them to be highly cited or wants to pander to their authors?
I also imagine there would be people out there who would become serial reviewers with the goal of making as much money as possible. I imagine the quality of their reviews wouldn't be great either.
For turnaround time, I would probably say yes. Right now it's typically to have several months to review a journal paper. And there's really not much a journal can do if a reviewer is late other than not ask them to review again. Even if reviewers were paid, I suspect it would be difficult from a budgetary perspective to difficult to have serial reviewing be lucrative with the possible exception of adjunct faculty who unfortunately are typically very underpaid.
Wait time is different. It is due to the editor overwhelmed, and the difficulty to find reviewers and also their typesetting made in foreign country with cheaper labor.
I have had a few papers submitted during the pandemic and the time limits given to the reviewer and the time the reviewer actually takes varies drastically. For example, one of my papers was under its second round of peer review for nearly four months. I have colleagues in similar positions with their papers as well.
I do think a monetary incentive would speed up the peer review process if the pay rate was cut for late review submissions.
And you say no I don't have time or your president said horrible things and you didnt distance yourself from him (MDPI) and they come back begging you to still review or give them the name of someone who could. Quality of reviewing has been ridiculous these last couple of years for this reason, and they start to go lower in the ranks. Many professors ask their grad students to review papers for them (so they benefit for it in their performance reviews while not doing the work)
So, if the problem is that scientists lack motivation to peer review, I think having peer review work recognized as important in tenure-and-promotion would probably be far more motivating than whatever hourly rate they'd end up paying.
Article authors are generally not paid! Yet there is no lack of motivation to write articles.
How to motivate to do peer-review well instead of perfunctorily is another question.
Article authors often pay as well. Either directly because the journal is a science-tabloid or because the authors want to make their work accessible to all. Or indirectly through the indirect cost the universities are taking from grant agencies to pay for their confidential agreements with publishers. Try to do a FOIA of a state university to get that information you will see how hard they will fight the fact that they get scammed.
Authors write because this is how you justify that you are really working, be it for your university, department or your grant agency. Also you want others to build upon your work, but academics would publish a lot less if it was not for the huge pressure from funders and employers (increasing numbers is all that count for many of them).
Peer review is also essential to the production of research articles. And is done largely by the same people who are writing articles; but they are incentivized non-monetarily to write articles, but not as much to review.
To restate my point perhaps more clearly: If reviewing were rewarded/expected as part of one's role at a university, department, or grant agency similarly to authorship, that would do far more to incentivize it than paying a relatively modest amount for a review.
I would argue that article authors generally are paid indirectly by the institution which employs them. Most scientific publications are authored by academics or members of research labs who are specifically paid to publish. Of course, pay is not directly tied to each publication, but that is what most authors are paid for. At least in my institution, contributing to the peer review process is also an expected part of the process to achieve tenure. But the incentive is certainly far less than the incentive to publish.
I think the problem with this is that it can encourage retaliation. If I know you rejected my paper, I might be incline to reject yours. Now, a nobody like me probably can't get away with doing that to someone with an established reputation. But the other way around is certainly possible.
Hunh, this is a great idea. Rejected articles should be published as rejected, and all reviews should be published. I guess this is a bit like the open review model, like The Cryosphere uses, except I think they hide papers after the discussion period if they aren't accepted.
How about making reviews available under the name of the reviewer. This means good reviewers can get respect of their reviewees, and reviewers are incentivized to write with slightly more consideration.
Similarly, universities should value good reviewers more. Which probably means reviewers need to get more prestige, so universities get prestige for having good reviewers.
This is something that is pushed by some people. With many journals I review for however, I am able to see otger reviewers reviews and I suppose they would be ashamed to have their names to be associated with them (extremely short and with generoc sentences that looks like they didnt even read the paper or just plainly agressive). I've heard many arguments against open review, but the most common is that people are afraid of retaliations if they criticize a work. I understand that people have retaliated to me after comments on PubPeer. But I still think we should do open reviews, that doesnt mean that journals that make people pay for publishing or reading should pay reviewers. We work for free get 0 reward... It is a bit like asking a designer to work for you, and tell them that it is to help the world of design and thank you no you can't sign...
Is there a way this can be made double-blind, where the Journal doesn't know the reviewer and the reviewer doesn't know the Journal? Can something akin to Mechanical Turk be utilized? Just trying to think of ways to have paid reviews without incurring all the negatives so typically associated with that arrangement.
I’d rather just reduce the number of for profit journals. Getting paid $450 seems to be against open access as open journals can’t afford this.
Also the motivations are different when pay is involved. I will donate time to peer review, but $450 is not enough to pay me. So I would be more likely to review as a volunteer than an employee.
I agree we don't need the publishers anymore. We have preprint services (arxiv and others), open review systems (pubpeer), sharing platforms (osf, zenodo, dataverse). You have Free software to handle all the manuscript lifecycle (there are a few cant remember the names). We keep the publishers because they still have some prestige and they are blackmailing university libraries. Now there is Scihub to go atound that, but of course there are ethical and legal reasons associated with its use so in my opinion thats not a long term alternative.
I want to contend that to most non-tenured professors (outside of the States at least?) 450$, even if not enough as pay either would be a nice extra from time to time.
I like money and $450 is a nice amount (hey, free PlayStation). But my employer doesn’t allow outside payments so I couldn’t accept.
And while I like money, I fee a duty to review a few papers a year, even if they are not good, or boring, or whatever. If I was being paid, I would be more likely to skip those papers because I no longer feel a duty.
This may be a similar motivation described in an experiment that showed how late pickups went up when the daycare introduced late fees. [0]
I think that many in academia are underpaid and should have more money. I just think that adding review fees are not the way.
In my (well, former) field you can only review papers if you are affiliated with a research organisation, so peer reviewers are necessarily in academia.
Can you share some of these non profits? I work at one who used to publish a journal for decades, but Sage has handled it in more recent memory. We make our salaries through grants and a library product. But I am not aware of others doing similar work. Partially out of ignorance.
If that programmer doesn't work for your company and you want them to do it in a timely manner, then yes, I think it's reasonable to do so. For open access publications, you could argue that reviewing is similar to working on an open source project, but open access is still the exception, not the norm.
Well, I much more like the question: Should authors pay for their publications? There is a common pun: As a scientist, you pay two times for an article: The one time you publish, and the other time when you subscribe to read your published article. And this is only to access the article as a PDF in the web. There is no print copy or so involved in that payment.
You usually have three models:
- pay for publish and read: big tabloid journals, large medical maf... associations. Cost to publish is usually below $1000, cost to read is $5-$20 a page
- pay for read, free to publish: most journals from the big publishers. Cost is between $5 to $20 a page.
- pay to publish, free to read: the model of most Open Access journals. Some big publishers are proposing that as an option as well. Cost of that option is usually $3000 to $5000 per article.
This depends on the field. I have never paid to publish and I doubt I ever would. (This does not count however the conference expenses typically required of an author to present their paper. But attending a conference has significant benefit to me personally, so I don't really see it as a direct cost of publication.)
Bring in a culture of paying peer reviewers, and suddenly authors (or the learned society) will be under more pressure to get more grant money to pay the new publication costs. Also, as someone who works in academia on an occasional basis, taking years off from funded positions at a time, I appreciate being able to write the occasional article on my own time and get it published without paying hundreds and hundreds of euro out of my own pocket -- fields with pay-to-publish traditions make it hard for non-permanent faculty to contribute.