How to play: Some comments in this thread were written by AI. Read through and click flag as AI on any comment you think is fake. When you're done, hit reveal at the bottom to see your score.got it
As an academic and regular submitter to arXiv, this is an eminently sensible policy. I like that it's per-submitter and not per-author, that means that the big labs with large collaborations shouldn't be terribly put out (more authors = more submission budget).
I hope something like this could also be adopted for some of our larger conferences - the absolute limits on co-authorship are what seem to cause the most grumbles.
It is the usually that LHC experiments (and HEP in general) secretariat office is the one handling the submission to arXiv and journals per policy (avoid people conflicts and managing it centrally because the papers under the name of the collaboration not individuals. Sometimes papers can also be submitted by author directly but not the physics results papers and papers which is signed off by the whole collaborations (computing and ML stuff mainly).
I would think arXiv can work something out for these cases.
Not for any neutrino experiment I know of. There are paper committees that govern publication but the actual preprint submission to arXiv is always done by one of the authors.
> I would think arXiv can work something out for these cases.
Yes, I also expect special cases would be defined.
> In September of 2016, arXiv received 9,869 submissions. In September of 2024, arXiv received 20,569 submissions. This September, arXiv received 40,363 submissions, which in turn generated almost 9,000 support tickets for arXiv staff and moderators.
> arXiv now limits submitters to up to two submissions per calendar month, with a limit of three total active submissions at any given time.
It seems there is a lot of this kind of AI risk. I'm trying to find the right label for it. Something that was a common good, that worked at a human scale, is now made unviable because automation has pushed it beyond what it can handle and still be useful.
several authors including the creator of arxiv, Paul Ginsparg, foretell a future in which we "increasingly rely on status markers such as author
pedigree and institutional affiliation as signals of quality, ironically
counteracting the democratizing effects of LLMs on scientific production."
I think this is a very plausible outcome of a flood of preprints. Because there's only so much time in a day and are we going to spend it on papers by less established people who we do not trust? It's a pity. If the science is good what does it matter who produced it.
Alternatively perhaps robust AI powered review pipelines eventually come into being. At that point the system as a whole would resemble a semi-supervised GAN setup optimizing for scientific paper quality. It might end up producing good results. (An absolute nightmare for the year or two leading up to that though.)
Some government should fund a preprint server where the submissions are screened by open weight models. Ultimately AI prescreening is far more scalable than human prescreening.
Thus biasing submissions' language and content towards those most palatable to those open-weight models, creating further incentives to generate articles wholesale with them, and incentives against publishing null findings
We ran this experiment with email around 2003. Bayesian filters worked great until spammers started salting messages with random dictionary words and book quotes. An open-weight screener is a public oracle: you just resubmit until it passes. Costs the attacker nothing.
This applies also to things at the human scale, when some wider group gets access to a common good.
Internet forums when internet access was only available to college students vs Internet forums today (called Eternal September, September being the month when new Freshman got access to the internet, Eternal September being when everyone in the developed world did)
Close, but classic commons stuff assumes the harm scales with how many people take a share. Here a single scraper can hammer an endpoint harder than every human reader combined. I've run a small crawler against a public API and hit limits within minutes. Rate limits were built for people, not for one script.
Well, somewhat. it's the same underlying behavior of individuals/organizations seeking advantage, leading to overuse and breakage. ..and, similarly, the resolution is in individuals backing public will. that is, someone (arxiv) takes an interest in the commons (scientific paper publishing) and curates/controls a segment of it, actively preventing abuse.
The thing that used to ration it was effort, and effort was free. We ran a small free API once, and within a year 90% of our bill was one scraper. Nobody planned for that. You end up putting up logins and rate limits, and the casual human user pays for it.
> There is also a marked increase in dense, AI-written papers. AI tools are making it easy for authors to flood arXiv and other repositories with these low-value papers.
You can't, not reliably. Any detector you build gets a false positive rate, and then someone's legit paper gets banned at 3am on a Friday and you're answering the appeals queue. Rate limits are the boring fix that actually works.
Arxiv papers are not supposed to signify anything. The only reason that people may derive some kind of career benefits is because Google Scholar indexes it and counts it towards the h-index etc. All that metric-based career-optimization going on is quite broken anyway, so this is fixing things from the wrong end as well. (The better end: stop hiring and promoting people based on Arxiv paper counts, so there is no incentive to flood it)
The hosting cost side I can understand, though they received quite some funding recently, but I assume it's not going towards actually running the site, but to who knows what broader impacts and so on.
Moderation for Arxiv is a silly idea. It already shouldn't be taken as a quality signal that something is able to be up on Arxiv. Obviously they should remove illegal stuff, but beyond that, requiring moderation is a misunderstanding of their role and reason for popularity.
If hosting is too expensive, I guess an alternative aggregator could also arise. With just metadata and a hash of the pdf that can be hosted anywhere, and as the sumbitter, if you move the file, you can change the URL.
The main reason for Arxiv's existence is the timestamping and the easy referencing. (Though I admit that the stable hosting is also a pretty important part, but they mention moderation effort as the reason, not the hosting costs.)
Academia is losing sight of the forest for the trees, can't see more than an arm's length ahead of their noses.
> Moderation for Arxiv is a silly idea. It already shouldn't be taken as a quality signal that something is able to be up on Arxiv.
It sounds like you're advocating for arXiv to become viXra. On https://vixra.org/all/2610 the second paper is currently "Correct Interpretation of the Great Discoveries in Particle Physics: I. Reconsidering the Higgs Boson via Vedic Vortex Structure", followed by a paper arguing that classical electrodynamics is bogus, "Gauss’s Flux Theorem Does not Hold in Time-Varying Electric Fields", and then "On the Boundary Problem in the Origins of Matter, Life, and Consciousness".
I don't think arXiv will continue to receive "quite some funding" if this is what it contains.
Vixra suffers from the "Witch Hunt problem" because Arxiv exists.
If Arxiv was the only game in town and it allowed everything, the result would be more like GitHub or Substack. That is, you wouldn't be able to trust everything hosted there, and a lot of it would be low-quality promotional garbage, with filtering moved to a different layer.
You could do some kind of karma system for example, where the prestige of your institution, your own karma, number of citations, papers published in high impact-factor journals etc would affect karma weight. You could have journals be like "awesome-x" lists on Github, with authors who have already been published there able to endorse other papers. I believe the mathematics community is trying that.
> You could do some kind of karma system for example, where the prestige of your institution, your own karma, number of citations, papers published in high impact-factor journals etc would affect karma weight.
The story that academia likes to tell itself about itself is that it is skeptical and iconclastic and objectively evaluates each piece of science on its own virtues and it doesn't matter who says it. A wrong thing said by someone famous is still wrong, and truth spoken by a nobody is still truth. Of course reality is a bit further from this ideal. But outright admitting it would be hard.
It's a bit like the "in America, anyone can become president!".
If you can't understand Vedic Vortex Structure without such karma system, your understanding will be bad anyway. Even legitimate publications are easy to misinterpret and sensationalize, social dynamics affects it too.
This is always the fate of explicitly "alternative" platforms, because only those people go there who are pushed out of the mainstream, who will tend to have some serious deficiencies, otherwise they'd try to be on the mainstream platform. Same with unmoderated social media turning into far-right places. Once a place obtains this type of reputation, it will be even more repellent for anyone not like that who wants to make it clear they are not like that, leading to a cascade, where the thing becomes a hermetically isolated "radioactive" place.
By the same principle, someone in the early 2000s could reasonably say "only weirdos date online", but the mainstream can shift also.
Arxiv is the principal forum for publishing mathematics research of quality. Journals serve a purely archival and bureaucratic purpose; almost no value is added by the refereeing process (probably more is lost) and journal publishing neither improves accessibility nor diffusion of results.
I agree fully that Arxiv is the principle publishing platform for mathematics. However, I have almost always gotten good value out of journal peer-review. It might just be that my sub-field is tightly knit, but I always receive thorough reviews with mostly good questions and suggestions for improvement. And I try to give the same when I do reviews of my own.
Even critical remarks are in the spirit of "You should do this better!", never the kind of gate-keeping bullshit I have seen in other fields.
Moderation for Arxiv is a silly idea. It already shouldn't be taken as a quality signal that something is able to be up on Arxiv. Obviously they should remove illegal stuff, but beyond that, requiring moderation is a misunderstanding of their role and reason for popularity.
Arxiv papers are regularly cited and assumed to be suitable for peer review. So some standards need to be maintained . it's not a free for all.
How about reading before citing? The bar was already very low for getting into Arxiv. If you used that as a quick decider of whether the work is good and should be trusted, you were already doing it wrong.
Who does the moderating, though? Citations to arXiv preprints already happen without anyone vouching for them. If a gatekeeping layer gets added, does it catch more junk than it creates false rejections of odd but legit work? Has anyone actually measured what the current endorsement system filters out?
Agree that it basically shouldn’t be necessary. But unfortunately Arxiv isn’t able to change the bad incentive structure that the broader hiring ecosystem has set up, right?
If academics and their committees actually deeply considered the scientific qualities of applicants, gaming the numbers would not matter. So I just find it hard to blame AI slop instead of the broken process that academia has settled on.
Part of the reason that the value of Arxiv papers has gone up, is that it's pretty well known that the conference review system is already quite broken and random and lazy and superficial. So being rejected from there is not a good reason to ignore a paper, so conference-rejected Arxiv-only but good papers are a regular occurrence. If conference review had better signal, arxiv could have zero signal.
The other issue is the time delay. Conferences have stopped being an actual place to learn about new work, the way it had been 10+ years ago. Now it's all outdated stuff, and as a specialist you already know the important papers months before the actual conference, so the conference is a networking event basically. The real exchange of ideas moved to Arxiv and Github and social media, with its own problems, hype, algorithmic engagement optimization incentives etc.
But the pressure that shifted away from conferences to arxiv is continuing. The old guard was pearl clutching already by the erosion of conference rubber stamps as this big prestige. The newer ones are now trying to hold back the tide at the Arxiv line. But it will keep on moving faster and faster. And what is going to matter in the end is not how things used to be, but what delivers actual value. If the slop is slop, it will dwindle. If it starts to be actually good, all this will just turn into basically a dock-unions-opposing-automation story.
I think it is a sensible idea, as a matter of fact, I think journals should do it too, maybe even more strict. Because the hike in submission means more burden for the peer-reviewers -- and this work is done (so far) for free.
In theoretical computer science, at least, researchers plan their work to finish on three major deadlines a year (SODA, STOC, FOCS). STOC [1] is requiring all submissions to be uploaded to ArXiv before submission, and allows up to 5 submissions per author. I wonder if they'll change their policy or instead adopt an additional de-facto limit of two per submitter.
In my experience, ArXiv has become too slow to publish. My previous paper took weeks to be approved. It was a recommended path for journal submission (posting to a public repository and submitting a link), and after waiting a long time, I just posted it on hal.science instead, and it was up in a day.
Another one of mine has been on hold for 18 days and counting.
I think it is a great service to the community, but clearly they are having a scalability problem.
Have you contributed volunteer moderator time? Ultimately, that is the most important resource being strained on ArXiv by large numbers of submissions.
Nice writeup and the policy is a step in the right direction. I suspect simple rate-limit measures like this will be a big improvement.
I suspect a logical conclusion Arxiv and elsewhere may be an identity management system with an aggressive filter, and shared blacklists. I suspect that classifying people as spam/slop-submitters, then banning them (or whatever identity they used; name, email, name + organization etc), applying incremental rate limits over the general one, may be required.
If the queue size grows beyond volunteer capacity then volunteer time becomes a scarce resource. In my experience (in other organizations), more likely than out and out block listing is aggressive prioritization. It takes sensible design to determine which papers get to consume moderators' time first.
Then, if someone posts rejected papers regularly and is deprioritized, it naturally follows that they may try to submit three more... and get stuck naturally. There is no need to ban them. They will still get a response, and they will get a fair shake - when there is time.
Since the joint truth of society now gathers on ArXiv it's interesting to see how we can revamp the efforts, since many of us are gathering up opposing views, setups and research gathered some other alternatives here in the comment section to check out!
They wrote "we are seeing a massive transformation in how researchers communicate their results," but they meant "how slop generators submit their slop".
The world will continue to be overwhelmed by people using AI to produce garbage. AI unlocks a treasure trove of new resources to be exploited. This is true even if AI can sometimes be useful.
There may be great researchers who work on several papers in a month, but even with this rule they can just split them and submit to arxiv, so I think it is very reasonable. AI slop is everywhere now (and also increasing), and I think this fast and forceful action by arxiv is effective at stopping this wave of slop.
It doesn't seem humanly possible to produce >2 "high SNR" papers per month. If you have a larger batch of related papers, it's easy to spread submissions over multiple months. You should probably be doing that anyway, because comments on one paper may affect the others.
Do you have a concrete example of a high SNR researcher who will be limited by this policy? Aka one who is submitting more than two high quality papers per month?
I hope something like this could also be adopted for some of our larger conferences - the absolute limits on co-authorship are what seem to cause the most grumbles.