ArXiv Trackback Policy
Ever since the trackback system at the arXivs was announced, it was clear that, sooner or later, a controversy would erupt. And, indeed, one has, surrounding the trackbacks of well-known 'Net personality, Peter Woit.
From the beginning, it was made clear that trackback privileges would not be open to all and sundry. The arXivs are a vehicle for communication between research scientists. Not everyone can have their papers appear on the arXivs. Similarly, not everyone would be able to have their trackbacks appear.
Woit has loudly protested the decision not to accept his trackbacks, and the discussion has spread elsewhere in the blogosphere. I have refrained, up till now, from commenting publicly because as a member of the arXiv Physics Advisory Board, I feel very constrained about what I can say publicly, either about the specifics of the case at hand, or about the internal deliberations of the Advisory Board.
But one thing became clear in the discussion over at Cosmic Variance. There’s a lot of confusion about the trackback policy. Some of that confusion was deliberately sown by people with an axe to grind; some of it was the unfortunate result of the less than transparent process under which the policy was developed.
So, what I’m going to do is try to explain the thinking that went into the policy, and then solicit your feedback.
Before plunging into details of policy, we ought to ask what the results of that policy have looked like. Overall, in its few months of operation, the system has worked quite well. There are several dozen blogs/websites from which we currently accept trackbacks. Collectively, they’ve generated several thousand trackbacks. The websites span the full range of fields represented on the arXivs, from high energy theory to condensed matter physics to quantum computation to astrophysics to computer science to quantitative biology. In high energy theory (in which I’ll include hep-th, hep-ph, gr-qc and hep-lat), there are about a dozen websites. In terms of trackbacks generated, the most prolific among them is John Baez’s This Week’s Finds, which has generated more trackbacks than all the other high energy theory sites combined. Trailing rather badly behind are this site, The String Coffee Table and all the rest. Overall, the most prolific site is NetAdv, which features review articles on a wide variety of fields of physics. It has generated more trackbacks (nearly 2400) than all the other sites, in all fields, combined.
Those sites with the largest number of trackbacks are, of course, those which have been around the longest. They may not retain their leads, going forward. But they are indicative both of the diversity and of the generally high quality of the trackbacks in the system.
Before talking about the trackback policy, we should start with the acceptance policy for papers. For a paper to be accepted to appear on the arXivs, it must go through a two-stage filter.
- The author must be an approved submitter, usually through having been endorsed.
- Each paper from an approved submitter must be accepted by the moderator for that section of the arXiv.
For a variety of practical reasons, it was decided that it would be infeasible to moderate each individual trackback. Trackbacks would go through just a single stage of filtering. If the author (or, more precisely, the author’s website) is approved, then (all of) his trackbacks may appear.
But what should the criteria for acceptance be?
One’s first thought is: why not use the same endorsement mechanism used for paper submission? Unfortunately, the experience of the moderation system is that endorsement is not a terribly high barrier to entry. Some endorsers are rather loose in endorsing people to submit papers and one can only imagine that they would be even looser in endorsing people to submit trackbacks. In the case of papers, the second-stage filter of moderation is clearly necessary. But we had already decided that there would be no such second stage in the case of trackbacks.
It is also vital to have a reasonably objective standard. “This looks like an interesting weblog.” was not going to be a workable criterion. Nor would any number of other subjective criteria.
The solution which was adopted, in the end, was that trackbacks would be accepted if they come from active researchers. It’s not particularly hard to figure out who’s an active researcher: just look at their publications. Exactly what level of activity counts as “active” is an issue. Wherever you draw the line, there will be borderline cases that require a judgement-call. But in most cases, the decision should be (and, indeed, has proven to be) straightforward.
The “active researcher” criterion is not perfect. A young graduate student, for instance, may not yet have much of a publication record but might, nonetheless, have useful things to say.
The decision whether to accept trackbacks from a given site is not immutable. In the case of a graduate student who started blogging well before she started publishing, it’s easy to start accepting her trackbacks once she has a few publications and has established herself as a researcher.
Conversely, nothing prevents us from stopping accepting trackbacks from previously acceptable sites if the author is posting defamatory, plagiarized, or otherwise unsuitable material. The same holds true for the ability to submit papers to the archives.
But, in the main, since trackbacks are an adjunct (a useful adjunct, but still an adjunct) to published papers, it’s not unreasonable to restrict trackback privileges to those who are actively engaged in research, as evidenced by their published papers1.
That’s the trackback policy, as it’s currently constituted. Peter Woit’s publication record doesn’t put him anywhere close to “active researcher” status. So, without some radical change in our criteria (or a radical change in his level of research activity), his trackbacks are not accepted.
Generally, there has been satisfaction with the trackback system, but some on the Board have expressed the feeling that the existing trackback acceptance policy is too lax. The scientific quality of commentary in the blogosphere is far too low, and we should either tighten our acceptance criteria or shut the whole system off.
I don’t know that I see a good way to tighten our acceptance criteria, so if it becomes the consensus that the trackbacks accepted under the current system are of generally too low a quality, it may indeed be that the only solution is to simply shut the system off.
I’d now like to turn this over to you, the readers, for discussion. I would like to hear suggestions for improving the trackback acceptance policy, or the trackback system at the arXivs generally. Or perhaps you agree with those on the Board who think this whole blogosphere thing is a waste of electrons and the trackback system should simply be shut down.
But first, some ground rules.
- Since I’m constrained from talking about the specifics of the Woit case (beyond what I have said above), so are you. There will be no pleading of Peter’s case here. Nor will attacks on Peter be tolerated. In both cases, such comments will be swiftly deleted. We will restrict the discussion to matters of general policy. And there’s very little about the specifics of this particular case that need to or ought to appear in such a discussion.
- If you want to propose an alternative trackback acceptance policy, remember the design criteria:
- It needs to be website-wide; we’re not going to start moderating individual trackbacks.
- It needs to be reasonably objective. Subjective criteria will just lead to endless controversies.
- It needs to be at least as restrictive as (maybe more restrictive than, if you want to satisfy the trackback-sceptics) the current policy.
- It’s probably not worthwhile trying to pin down what the precise boundary between “active researcher” and John Q. Blogger should be. As I said, wherever the precise line is drawn, there will always be borderline cases. But with any reasonable choice of where to draw the line, Woit isn’t one of those borderline cases.
- Finally, we are not going to turn this into a discussion of “censorship” by the arXivs. If you are banned from posting papers to the arXivs, you may have a legitimate grievance. But this is not the place to air such grievances. Again, such comments will be deleted.
So that’s it. I’ll don my asbestos underpants and let y’all have at it …
Update:
Paul Ginsparg has sent me the latest stats from the trackback database. There are currently 5132 trackbacks from 51 approved sites. In high energy theory, the sites with 30 or more trackbacks are:| Blog | # trackbacks |
|---|---|
| This Week’s Finds | 731 |
| The String Coffee Table | 300 |
| Musings | 268 |
| PhysComments | 83 |
| Luboš Motl | 56 |
| Cosmic Variance | 33 |
1 It probably goes without saying that you couldn’t use the same criterion to decide who can submit papers to the arXivs. That would be a Catch-22.
Re: ArXiv Trackback Policy
It seems to me that the crucial question here is not “Is the commenter an active researcher?” but rather “Does the site offer technical discussion of the subject matter?”.
“Active researcher” seems very difficult to implement in practice. You end up having to make some rather arbitrary definitions, which are then immediately supplemented by exceptions for retired faculty, young grad students, etc,… What if Matt Nobes starts blogging again? What if Grothendieck decides he wants to speak up?
“Technical discussion”, on the other hand, seems fairly easy to implement. Are there equations? Error bars? Is the language appropriately high level? Does the commenter consider the specific details discussed in the paper, or is the commenter offering only generalities?
And as an added bonus, this sort of policy would encourage people to raise their standards for scientific discourse, rather than igniting flamewars about credentials.
–A.J.
ps. I hope the underpants hold up.