ArXiv Trackback Policy

March 5, 2006

ArXiv Trackback Policy

Ever since the trackback system at the arXivs was announced, it was clear that, sooner or later, a controversy would erupt. And, indeed, one has, surrounding the trackbacks of well-known 'Net personality, Peter Woit.

From the beginning, it was made clear that trackback privileges would not be open to all and sundry. The arXivs are a vehicle for communication between research scientists. Not everyone can have their papers appear on the arXivs. Similarly, not everyone would be able to have their trackbacks appear.

Woit has loudly protested the decision not to accept his trackbacks, and the discussion has spread elsewhere in the blogosphere. I have refrained, up till now, from commenting publicly because as a member of the arXiv Physics Advisory Board, I feel very constrained about what I can say publicly, either about the specifics of the case at hand, or about the internal deliberations of the Advisory Board.

But one thing became clear in the discussion over at Cosmic Variance. There’s a lot of confusion about the trackback policy. Some of that confusion was deliberately sown by people with an axe to grind; some of it was the unfortunate result of the less than transparent process under which the policy was developed.

So, what I’m going to do is try to explain the thinking that went into the policy, and then solicit your feedback.

Before plunging into details of policy, we ought to ask what the results of that policy have looked like. Overall, in its few months of operation, the system has worked quite well. There are several dozen blogs/websites from which we currently accept trackbacks. Collectively, they’ve generated several thousand trackbacks. The websites span the full range of fields represented on the arXivs, from high energy theory to condensed matter physics to quantum computation to astrophysics to computer science to quantitative biology. In high energy theory (in which I’ll include hep-th, hep-ph, gr-qc and hep-lat), there are about a dozen websites. In terms of trackbacks generated, the most prolific among them is John Baez’s This Week’s Finds, which has generated more trackbacks than all the other high energy theory sites combined. Trailing rather badly behind are this site, The String Coffee Table and all the rest. Overall, the most prolific site is NetAdv, which features review articles on a wide variety of fields of physics. It has generated more trackbacks (nearly 2400) than all the other sites, in all fields, combined.

Those sites with the largest number of trackbacks are, of course, those which have been around the longest. They may not retain their leads, going forward. But they are indicative both of the diversity and of the generally high quality of the trackbacks in the system.

Before talking about the trackback policy, we should start with the acceptance policy for papers. For a paper to be accepted to appear on the arXivs, it must go through a two-stage filter.

The author must be an approved submitter, usually through having been endorsed.
Each paper from an approved submitter must be accepted by the moderator for that section of the arXiv.

For a variety of practical reasons, it was decided that it would be infeasible to moderate each individual trackback. Trackbacks would go through just a single stage of filtering. If the author (or, more precisely, the author’s website) is approved, then (all of) his trackbacks may appear.

But what should the criteria for acceptance be?

One’s first thought is: why not use the same endorsement mechanism used for paper submission? Unfortunately, the experience of the moderation system is that endorsement is not a terribly high barrier to entry. Some endorsers are rather loose in endorsing people to submit papers and one can only imagine that they would be even looser in endorsing people to submit trackbacks. In the case of papers, the second-stage filter of moderation is clearly necessary. But we had already decided that there would be no such second stage in the case of trackbacks.

It is also vital to have a reasonably objective standard. “This looks like an interesting weblog.” was not going to be a workable criterion. Nor would any number of other subjective criteria.

The solution which was adopted, in the end, was that trackbacks would be accepted if they come from active researchers. It’s not particularly hard to figure out who’s an active researcher: just look at their publications. Exactly what level of activity counts as “active” is an issue. Wherever you draw the line, there will be borderline cases that require a judgement-call. But in most cases, the decision should be (and, indeed, has proven to be) straightforward.

The “active researcher” criterion is not perfect. A young graduate student, for instance, may not yet have much of a publication record but might, nonetheless, have useful things to say.

The decision whether to accept trackbacks from a given site is not immutable. In the case of a graduate student who started blogging well before she started publishing, it’s easy to start accepting her trackbacks once she has a few publications and has established herself as a researcher.

Conversely, nothing prevents us from stopping accepting trackbacks from previously acceptable sites if the author is posting defamatory, plagiarized, or otherwise unsuitable material. The same holds true for the ability to submit papers to the archives.

But, in the main, since trackbacks are an adjunct (a useful adjunct, but still an adjunct) to published papers, it’s not unreasonable to restrict trackback privileges to those who are actively engaged in research, as evidenced by their published papers¹.

That’s the trackback policy, as it’s currently constituted. Peter Woit’s publication record doesn’t put him anywhere close to “active researcher” status. So, without some radical change in our criteria (or a radical change in his level of research activity), his trackbacks are not accepted.

Generally, there has been satisfaction with the trackback system, but some on the Board have expressed the feeling that the existing trackback acceptance policy is too lax. The scientific quality of commentary in the blogosphere is far too low, and we should either tighten our acceptance criteria or shut the whole system off.

I don’t know that I see a good way to tighten our acceptance criteria, so if it becomes the consensus that the trackbacks accepted under the current system are of generally too low a quality, it may indeed be that the only solution is to simply shut the system off.

I’d now like to turn this over to you, the readers, for discussion. I would like to hear suggestions for improving the trackback acceptance policy, or the trackback system at the arXivs generally. Or perhaps you agree with those on the Board who think this whole blogosphere thing is a waste of electrons and the trackback system should simply be shut down.

But first, some ground rules.

Since I’m constrained from talking about the specifics of the Woit case (beyond what I have said above), so are you. There will be no pleading of Peter’s case here. Nor will attacks on Peter be tolerated. In both cases, such comments will be swiftly deleted. We will restrict the discussion to matters of general policy. And there’s very little about the specifics of this particular case that need to or ought to appear in such a discussion.
If you want to propose an alternative trackback acceptance policy, remember the design criteria:
- It needs to be website-wide; we’re not going to start moderating individual trackbacks.
- It needs to be reasonably objective. Subjective criteria will just lead to endless controversies.
- It needs to be at least as restrictive as (maybe more restrictive than, if you want to satisfy the trackback-sceptics) the current policy.
It’s probably not worthwhile trying to pin down what the precise boundary between “active researcher” and John Q. Blogger should be. As I said, wherever the precise line is drawn, there will always be borderline cases. But with any reasonable choice of where to draw the line, Woit isn’t one of those borderline cases.
Finally, we are not going to turn this into a discussion of “censorship” by the arXivs. If you are banned from posting papers to the arXivs, you may have a legitimate grievance. But this is not the place to air such grievances. Again, such comments will be deleted.

So that’s it. I’ll don my asbestos underpants and let y’all have at it …

Update:

Paul Ginsparg has sent me the latest stats from the trackback database. There are currently 5132 trackbacks from 51 approved sites. In high energy theory, the sites with 30 or more trackbacks are:

High Energy Theory Sites with 30 or More trackbacks at arXiv.org
Blog	# trackbacks
This Week’s Finds	731
The String Coffee Table	300
Musings	268
PhysComments	83
Luboš Motl	56
Cosmic Variance	33

Also, for those commenters advocating an arXiv-sponsored blog, an arXiv-sponsored slashdot-style site, an arXiv-sponsored comment forum, …it’s not gonna happen. Please take the time to read my previous post on the rationale behind the trackback system, if you want to understand why.

¹ It probably goes without saying that you couldn’t use the same criterion to decide who can submit papers to the arXivs. That would be a Catch-22.

Posted by distler at March 5, 2006 11:38 PM

TrackBack URL for this Entry: http://golem.ph.utexas.edu/cgi-bin/MT-3.0/dxy-tb.fcgi/760

100 Comments & 5 Trackbacks

Re: ArXiv Trackback Policy

It seems to me that the crucial question here is not “Is the commenter an active researcher?” but rather “Does the site offer technical discussion of the subject matter?”.

“Active researcher” seems very difficult to implement in practice. You end up having to make some rather arbitrary definitions, which are then immediately supplemented by exceptions for retired faculty, young grad students, etc,… What if Matt Nobes starts blogging again? What if Grothendieck decides he wants to speak up?

“Technical discussion”, on the other hand, seems fairly easy to implement. Are there equations? Error bars? Is the language appropriately high level? Does the commenter consider the specific details discussed in the paper, or is the commenter offering only generalities?

And as an added bonus, this sort of policy would encourage people to raise their standards for scientific discourse, rather than igniting flamewars about credentials.

–A.J.

ps. I hope the underpants hold up.

Posted by: A.J. on March 6, 2006 2:16 AM | Permalink | Reply to this

Re: ArXiv Trackback Policy

“Technical discussion”, on the other hand, seems fairly easy to implement. Are there equations? Error bars? Is the language appropriately high level? Does the commenter consider the specific details discussed in the paper, or is the commenter offering only generalities?

To the contrary, by virtue of being inherently a judgement-call, it’s far harder to implement in a consistent and impartial manner.

Indeed, the qualities you would use as the basis for deciding whether to approve a site’s trackbacks vary, in my experience, rather dramatically from post to post on some sites. Some posts may be full of technical details; others might be rather cursory.

Unless we’re willing to moderate individual trackbacks, it’s hard to base an approval system on the content of individual posts.

And as an added bonus, this sort of policy would encourage people to raise their standards for scientific discourse, rather than igniting flamewars about credentials.

Raising the standards of scientific discourse in the blogosphere is certainly an laudable goal (we, here at Musings, strive for the highest standards of scientific discourse).

Flame wars are one of, but far from the only, thing dragging down those levels. But if those flame wars don’t generate trackbacks to the arXivs, then they are irrelevant to the discussion at hand.

On the other hand, as I said, there are those who argue that level of scientific discourse, probably even of the posts here at Musings, is too low for the arXivs to want to be associated with them — even at the arms-length afforded by the trackback system.

Posted by: Jacques Distler on March 6, 2006 2:57 AM | Permalink | PGP Sig | Reply to this

Re: ArXiv Trackback Policy

To the contrary, by virtue of being inherently a judgement-call, it’s far harder to implement in a consistent and impartial manner.

Yes, of course it’s a judgement call. So is “active researcher”; that’s the basis of all these complaints.

But asking for technical commentary keeps out the crackpots while still allowing a wide variety of commentary. Ask for more than that, and you have three choices: 1) make arbitrary distinctions, 2) make editorial decisions case by case, or 3) raise your standards very very high.

Personally, I can see the case for 3). And honestly, it’s not that hard to distinguish between the sort of technical commentary offered at String Coffee Table and Life on the Lattice, and the more general sort of commentary you find at Cosmic Variance or Not Even Wrong.

But if you want 1) or 2), you really must clarify: What are you asking for, beyond banning crackpots?

And out of curiosity, what have the current standards actually accomplished? Who is banned? Who is not?

Posted by: A.J. on March 6, 2006 2:28 PM | Permalink | Reply to this

Re: ArXiv Trackback Policy

“Technical discussion”, on the other hand, seems fairly easy to implement.

I disagree. Peter’s case is instructive here. The spectrum of publically aired opinion on this runs from “anti-science crackpot”, through “the same critique we’ve heard for years”, to “wrong, but worth listening too” and “a sharp critique, exposing major flaws in HEP” (those aren’t acutal quotes, just my interpretation of the opinions I’ve read). All those seem to be opinions held by “active” physicists. So how does one decide “technical” or not?

That being said, I think the active researcher standard is off the mark as well. The “active researcher” idea totally misses the possibility of an outsider, who may have a valid point, being seen by those who read arxiv trackbacks. Instead, this standard will only be physicists commenting on other physicists.

What if Matt Nobes starts blogging again?

Not likely :), but fortunately I’ve turned my blog over to an active researcher, to carry it forward.

Posted by: Matthew on March 6, 2006 3:09 AM | Permalink | Reply to this

Re: ArXiv Trackback Policy

I agree more or less with A.J. I think that the goal of the moderation should be to eliminate trackbacks to blogs that don’t discuss the paper seriously or make abusive comments.

To implement this efficiently, arXiv should use a complaint policy. Only the obvious trackbacks to spam sites (e.g. viagra advertisements) should be automatically blocked.

Now suppose that there exists blog that unfairly criticises papers, uses abusive language toward the authors, blocks comments from people defending the paper or does something else distasteful, then people should complain to arXiv. This will probably only happen in a very small prcentage of the trackbacks. Perhaps you’ll have one case per year. Then if the complaint is judged to be valid, trackbacks to the blog will be banned.

Let me give some motivations why you want to have a liberal policy regarding trackbacks and not just allow trackbacks from blogs of people who are experts in the field.

1) What matters is the discussion on the blog and if it is managed properly. If I have a blog and post about an interesting article on string theory, any expert is able to participate in the discussions. The level of the discussion doesn’t necessarily correlate with my knowledge about the subject.

2) Most scientists would like it if there preprints had some trackbacks to blogs managed and read by non experts.

Posted by: Count blis on March 6, 2006 11:17 AM | Permalink | Reply to this

Re: ArXiv Trackback Policy

I agree more or less with A.J

We aren’t in complete agreement, however: I lean towards frankly elitist standards, and you seem willing to let everyone have a say.

And “one a year” seems very very low. Just ask any of the sci.physics.research moderators how much time it takes to deal with the crackpots.

Posted by: A.J. on March 6, 2006 2:36 PM | Permalink | Reply to this

Re: ArXiv Trackback Policy

A.J., I don’t think you would have many cranks posting trackbacks to arXiv articles. Usenet groups and message boards do attract cranks because they want to let the world know about their new ”theory of everything”. They usually aren’t interested in real physics articles at all.

Another thing: Perhaps one should allow the author of an arXiv preprint to block trackbacks to his preprint for whatever reason. So, Lubos Motl would probably remove trackbacks from Peter Woit’s blog to his article, but I would decide to keep them :)

Posted by: Count Iblis on March 6, 2006 4:35 PM | Permalink | Reply to this

Re: ArXiv Trackback Policy

Thank you for bringing this up. This seems a bit of a tempest in a teacup…but an important one. Effectively, as you explain it, the arxiv would like to create an ‘in-group’ and an ‘out-group’ to keep discussion both serious and scientific. One proposed division is that ‘active researchers’ be the considered the in-group.

Not a bad start. The aim is noble. The reasoning inspired. The motive, undoubtedly pure, public spirited, and unsullied by rent-seeking behavior. But let’s consider at least a few harmless thought experiments to test any exclusionary policy:

Would Watson or Crick have been active researchers to Erwin Chargaff before their big break? Would Alexander Grothendiek be allowed trackbacks as a blogger if he started his own site in his current state? How about Irving Segal as a cosmologist (rather than a mathematician)? Would John Nash be allowed to comment on game theory when not in a proper frame of mind? What about Papakyriakopoulos before Ralph Fox gave him legitimacy at Princeton? How about letting Teichmuller and Jordan share their views on national socialism (as well as moduli and commutative algebras). What if Streleski or Kaczynski wishes to start picking up research (and blogging) again? The Bogdanovs? Luca Turin? Kary Mullis? Ramanujan (pre Hardy)?

The trouble for your goup seeking to put up a velvet rope is that there are a disproportionate number of fascinating minds concentrated among the truly marginal cases. It’s like counting the proportion of stellar individuals among the tiny group of Harvard dropouts: it is, gloriously, somewhat higher than might otherwise be expected.

Alas, science isn’t a guild run garden party with a guest list. At its best, it’s more like a free-for-all rave where we all throw up our ideas to see which brainwaves stick to the walls.

Like it or not, the next big thing can come from anywhere and, more importantly, anyone. Creeps, cranks, gadflys, losers, felons, anti-semites, lunatics and [perhaps worst of all] non-tenure track faculty (horrors!) have all contributed to the edifice of science.

You will eventually face this problem, but the blog that started this issue rolling is an IDIOTIC test case. You have no idea how many top flight active researchers tune in to the blog that is motivating your post. Please, stop protecting us from such dangerous people. In the case of Dr. Woit, it makes it look like String theory dost protest too much.

Look. Science isn’t always pretty. Make a policy, announce it and stick to it with minimal squishiness. The worst thing you can do is resort to backhanded dismisals of people who are not ‘active researchers’.

Requiring a reputible PhD will solve most (though not all) of the problem. Then, just hope against hope that the next big idea comes from one of an A-list colleague and not a disgruntled outcast.

A Friend

Posted by: c niedman on March 6, 2006 3:10 AM | Permalink | Reply to this

Read the post Physics Catfight!
Weblog: Uncertain Principles
Excerpt: There's a kerfuffle in the physics blogosphere these days over the somewhat arcane issue of TrackBacks to posts on the ArXiV, the commual preprint server where researchers can post drafts of the papers that they have submitted to research journals...
Tracked: March 6, 2006 7:16 AM

Re: ArXiv Trackback Policy

To me it would already be helpful if somewhere there were a list of “allowed sites”. Not only because that would give one more ways to spend one’s idle time but it would also tell you if you are facing technical problems or you are not whitelisted.

Me for example, was in the past able to post trackbacks via atdotde (at least that was my impression, viz e.g. http://www.arxiv.org/abs/astro-ph/0509039 where the “trackback” link currently gives me a “redirection limit exceeded” error). But my most recent atempt failed: When entering the ping into Haloscan for http://arxiv.org/abs/hep-th/0602072 there was no error message but the trackback has not appeared up to now.

So, do I have problems with haloscan? Am I not active enough to post trackbacks (both paper- and blog-wise)? I don’t know. If there were I list of trackback enabled sites, I could tell.

Robert

Posted by: Robert on March 6, 2006 7:16 AM | Permalink | Reply to this

Re: ArXiv Trackback Policy

http://www.arxiv.org/abs/astro-ph/0509039 where the “trackback” link currently gives me a “redirection limit exceeded” error

One of the many bugs in the system.

To me it would already be helpful if somewhere there were a list of “allowed sites”.

Right now, as far as I understand, the only way to find that out is to have direct SQL-query access to the database. (The statistics I cited above are 2-month old statistics furnished by Paul Ginsparg. adotde was on the approved list.)

Add this to the TO-DO list.

Posted by: Jacques Distler on March 6, 2006 7:55 AM | Permalink | PGP Sig | Reply to this

Allowed Trackback Sites

$MathML-enabled post (click for more details).$

Is the list of “allowed sites” available, or is that still a TO-DO item? I’d wanted to note on EUREKA the sites for which trackbacks are allowed, but I wasn’t able to find that information at arXiv.org or elsewhere.

Posted by: Blake Stacey on December 10, 2007 3:39 PM | Permalink | Reply to this

Re: Allowed Trackback Sites

Is the list of “allowed sites” available

Not to my knowledge. The only web interface I know about is this one.

or is that still a TO-DO item?

I have no idea. I am not at all involved in the implementation, the management or the operation of the Trackback Service.

I can ask…

Posted by: Jacques Distler on December 10, 2007 3:56 PM | Permalink | PGP Sig | Reply to this

Re: ArXiv Trackback Policy

It is clear to me that the scientific level of discussions on the web is not quite there yet, but there is nothing inherent in the medium that prvents this from happening. I think the trackback system can be one step towards increasing the level of discussion and encouraging more active researchers to participate and offer their points of view. When this happens, and it does occasionally, one can get a glimpse of something indispensable. Hope then that trackbacks continue, and the current level of moderation looks good to me.

Posted by: Moshe on March 6, 2006 8:34 AM | Permalink | Reply to this

Re: ArXiv Trackback Policy

——-BEGIN SIGNED MESSAGE——- Hash: SHA1

Ah… I had written a long comment, but my browser crashed on me on i hit preview… sigh

Anyway, all i mean to say is that: The way it’s currently set, the system uses “paper count” to classify allowance into the trackback system. What i think is that you’ll need a more plural and diverse approach to this problem (given its very nature and setting – the ‘Net). Right now the system allows those folks that have a high ‘paper count’ with very little to say. OTOH, those folks with a lower ‘paper count’ but with more ideas to add and contribute to a given discussion, are left out.

I do realize that this 2nd category is the one that lies mostly on the “boundary” of the 1st one, but i think that an equally arbitrary set of rules can be used, such as: institution filiation (reputable university/college?), academic heritage (reputable advisor? postdocs?), allowed to publish on the arXivs, etc. (Note that these rules can be extended for different countries.)

I’m not claiming those to be a ‘good’ set of rules but it’s my opinion that they are as good as ‘paper count’. (Sure enough, my opinion may not be of great value… ;)

Also, although i’m not suggesting manual moderation, i don’t see why the following scenario would be ‘bad’: A given arXivs moderator is surfing the web on s/he bumps into an “interesting” website – it can be submitted to the criteria above and, eventually, added to the list of allowed trackbacks. (This is not to say that people should email requests to the moderators but, it’d be something like “Stumble Upon” in Mozilla. ;)

On a different token: “Thanks, Jacques, for making this subject available to public discussion and heling take care of the arXivs trackback system.” :) ——-BEGIN SIGNATURE——- Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFEBdVTwZPTkJ/pv0sRApSxAKClLQJ7elbAA52WTZiVT6za/7cz4wCcDho+ ycdop5Uhe3pFk/c4xzUFt7M= =u/2s ——-END SIGNATURE——-

Posted by: Daniel Doro Ferrante on March 6, 2006 9:42 AM | Permalink | Reply to this

Re: ArXiv Trackback Policy

One way to compromise would be to allow trackbacks for any blog that, say, 20 active researchers (by the current definition) endorse as one they read/comment on frequently.

Posted by: Tez on March 6, 2006 10:29 AM | Permalink | Reply to this

Re: ArXiv Trackback Policy

Dear Jacques,

I agree with those on the Board who advocate shutting down the trackback system.

At least in the high energy physics community, the arxiv is the primary tool for communcation among scientific researchers. And the central goal of the arxiv should be to maintain as high a level of communication among scientific researchers as possible. Frankly, most of the physics discussion on blogs is simply not at a high enough level to be useful for a researcher. (Your blog is one of the very few exceptions to this statement). Blogs are occasionally useful for providing pointers to papers I’ve missed or summaries of papers I haven’t read, but the vast majority of blog posts are simply not research-grade material. They are typically either too vague, personally biased, irrelevant or incorrect to be useful. This is not to say that blogs are worthless, but only to say that when I am in the mood to wallow in the muck of the blogosphere I am perfectly capable of doing so on my own without the help of an arxiv trackback system. Including trackbacks lowers the overall level of discourse on the arxiv, with very little benefit.

One last comment: the arxiv is neither a tool for communication to the general public nor an informal online discussion forum. Of course, these things are important and useful, but we should maintain a clear distinction between the arxiv as a tool for formal research-level communication, and the blogosphere, which is by its nature informal. If someone would like to set up a physics blog aggregator or something like that, this may be worthwhile. But adding blog trackbacks to the arxiv needlessly blurs the line between real research and informal discussion.

Thanks.

Posted by: Anonymous on March 6, 2006 1:00 PM | Permalink | Reply to this

Re: ArXiv Trackback Policy

Frankly, most of the physics discussion on blogs is simply not at a high enough level to be useful for a researcher. (Your blog is one of the very few exceptions to this statement).

This brings up a point that I probably should have emphasized. If you’re looking at this from the perspective of blogs (particularly those which happen to be the most popular and best-known in the blogosphere), you’re looking at it through the wrong end of the telescope.

What’s important here is the actual trackbacks at the arXivs. In our field (high energy theory), the vast majority of trackbacks come from John Baez (way ahead), followed by this blog and the String Coffee Table.

I’d call those three “high quality” sources. Wouldn’t you? All the other blogs combined produce only a miniscule fraction of the trackbacks in high energy theory.

They may be more entertaining to read, they may be vastly more popular, but they are mostly irrelevant, if you are trying to judge the “average quality” of trackbacks at the arXivs.

[That said, I’m not sure that the trackbacks from NetAdv actually add much value. But that’s another discussion…]

Posted by: Jacques Distler on March 6, 2006 1:31 PM | Permalink | PGP Sig | Reply to this

Re: ArXiv Trackback Policy

From the beginning, it was made clear that trackback privileges would not be open to all and sundry. The arXivs are a vehicle for communication between research scientists. Not everyone can have their papers appear on the arXivs. Similarly, not everyone would be able to have their trackbacks appear.

I’m not sure I entirely follow the basic premise of disallowing trackbacks from arbitary users. It’s pretty clear that limiting the material uploaded to on-topic items - i.e. scientific research papers - is necessary to make the arXiv useful - otherwise there would simply be too low a signal-to-noise ratio for anyone to bother using the service. On the other hand I’m not quite sure that the same thing applies to trackbacks. The primary use of a trackback (it seems to me) is to point people who are already interested in a paper toward discussion which includes some mention of the paper*. That discussion can be interesting even if it isn’t among professional scientists - for example I guess many people would be interested to know if their paper had been featured on Slashdot (assuming Slashdot sent trackbacks which, sadly, AFAIK, they do not). Indeed, I would say that enabling trackbacks in this way could be a very useful tool in gauging the public reaction to certian pieces of research.

I assume the objection to such a setup is that it would create a signal-to-noise problem? I tend to disagree - I think the number of trackbacks will be small whoever is allowed to contribute, but I’ll note that limiting the field to “active” researchers provides no guarantee of quality - after all there is a time-honored tradition of Nobel Prize Winners Saying Extremely Stupid Things About Fields Outside Their Area Of Expertise. There is nop reason to assume that won’t also apply to the discussion of arXiv papers in another (sub-)field by an “active” researcher.

Of course there’s no reason that all trackbacks have to be equal. One could, for example order trackbacks so that “active” researchers appear at the top of the list and everyone else is relegated to the bottom. I know physicists are generally bad at solving UI problems (at least, the ones who design astronomy software certainly are) but it doesn’t seem too hard to do well enough that it keeps the signal to noise low for everyone who wants to ignore the “little people”.

*Under this definition, there is no particular value in a trackback from a site which simply collates lists of papers. I suppose the current policy makes more sense to me if we assume that you’re viewing a trackback like a traditional citation. However, given the nature of the web I don’t think it makes sense to compare the two.

Posted by: jgraham on March 6, 2006 1:22 PM | Permalink | Reply to this

Re: ArXiv Trackback Policy

but it doesn’t seem too hard to do well enough that it keeps the signal to noise low for everyone who wants to ignore the “little people”

Should read:

but it doesn’t seem too hard to do well enough that it keeps the signal to noise high for everyone who wants to ignore the “little people”

Posted by: jgraham on March 6, 2006 1:25 PM | Permalink | Reply to this

Re: ArXiv Trackback Policy

Hi,

I would just like to throw an idea out here, that could perhaps resolve the blog/trackbacks issue.

Instead of having trackbacks to blogs, which are led by only a few researchers (who like and take time to blog), perhaps there could be directly on arXiv a “slashdot” (http://www.slashdot.org) style discussion forum attached to each paper published on the arXiv. Any researcher endorsed to publish on the arXiv could post comments to articles.

Moreover, instead of having to create some sort of “censorship rules” as it is necessary to do with the actual trackback system (whose blog should be endorsed, whose blog should not be), there could be again a “slashdot”-style system of “voting”, where readers can give a “rate” to the comments, according to their pertinence. Then each reader can set its own level of reading, that is for instance “I want to see only comments that have at least 8/10”, “I do not want to see any comments”, “I want to see all comments”, etc.

This kind of system introduces a self-sustainable “censorhip” system that does not (in principle) require the intervention of the administrators of arXiv. The whole research community, through the “rating” system, administrates the discussion forum.

This is even more true if only endorsed people (from arXiv) can post comments on the discussion forums. This sort of system is used on many websites at the moments, not only slashdot; websites that tend towards “open-publishing” but want to keep a certain level of pertinence.

I don’t know what people think about this idea, but I personally think it would be much easier to manage than the trackback system, and also it would give the opportunity to everyone in the scientific community (not only bloggers) to comment on published articles. That could lead to very interesting and stimulating discussions. That would definitely be a “new” way of doing research, but I think we should take advantage of the new possibilities opened up by the internet and our wonderful free publishing system which is the arXiv. Technically I don’t think it would be very difficult to implement, as it is already implemented on many websites (although I do not know how the arXiv works technically).

Best,
Vincent

Posted by: Vincent Bouchard on March 6, 2006 3:17 PM | Permalink | Reply to this

Re: ArXiv Trackback Policy

I am very much in favour of establishing new (likely web enabled) possibililties to give feedback to papers in addition to writing a new paper or a “comment on…” or just talking to the people who happen to be around when you fill your coffee mug in the morning (assuming European style where the new preprints arrive in the morning). The simplest version would be to make the names of refereees public who accept a paper. At least today (who knows where we’re heading) blogs are closer to the “ranting to office mates” end of the scale but trackbacks makes them more valuable. It would be bad if the trackback feature would be shut down because of this issue.

However general discussion boards for papers already exist and they don’t work too well as there is far too much noise. Blogs on the other hand provide a slightly more formal frame and at least in theory the author(s) have some credebility to lose.

Posted by: Robert on March 6, 2006 3:57 PM | Permalink | Reply to this

Re: ArXiv Trackback Policy

Jacques has asked that the specifics of my case not be discussed here, but then has gone on to announce that

“Peter Woit’s publication record doesn’t put him anywhere close to “active researcher status”.

For “publication record”, he embedded a link not to, well, my publication record, but to the list of my arXiv postings. He is well aware of how SPIRES works and knows that something much more accurate would be this:

publication record

Sure, most of these publications are from the 80s, when I was actively involved in work on a rather hot topic in lattice gauge theory. During the 90s I continued to be actively involved in research, but changed to thinking about quite different things, working on very unpopular ideas that no one else seemed to be working on and thus likely to scoop me. I won’t argue against anybody who tells me that I should have written more of this up as I went along, but I will strongly object to anyone who claims that those years were not spent doing “active research”.

The fruits of this were written up in one moderately long 56 page paper and posted on the arXiv in June 2002 as hep-th/0206135. This was never submitted for peer review, since I didn’t much see the point. I have no doubt I could have found a journal to accept it, but don’t see what that would have accomplished.

This history according to Jacques clearly doesn’t make me an “active researcher”, but he gives no indication why this is the case. If I had submitted the paper for peer review and it had been accepted would I now be an “active researcher”? If I submit it somewhere tomorrow and it is accepted will I all of a sudden become an “active researcher”? Is the problem that it is only one paper? What if I had broken it up into three papers and posted those on the arXiv, would I now be an “active researcher”? What if these three papers were posted over a period of years, what if they were peer-reviewed, etc?

Since 2002 I’ve continued to work actively in this area, and have given talks about this work; slides for talks at a conference in 2003 and at a Dartmouth colloquium in 2004 are available at my web-site. Do those materials count as evidence that I’m an “active researcher”? I have a couple manuscripts in preparation that I’m not yet happy with, but hope to finish and post on the arXiv later this year. If I do this will I become an “active researcher”? Can I put all this together in one long paper, or to become an “active researcher”, will I have to break things up into multiple pieces? How many pieces should I break my manuscript into in order to become an “active researcher”? Will they need to be peer reviewed?

During 2003-2005 I taught various graduate courses here in the Columbia math department, and this experience involved a great deal of involvement with research level math and physics. I put a huge amount of time into writing up graduate course notes that explain the geometric approach to representation theory and how this is related to geometric quantization and physics. Much of this I think is rather original, at least in presentation. It’s available on my web-site, judge for yourself. Does doing this make me an “active researcher”, even though it’s not on my “publication record”? I also spent a lot of time working on a graduate quantum field theory course for mathematicians. Again this involved a sizable amount of research level work, since much of this was original in one way or another. I now have a hand-written notebook filled with these notes, waiting for me to find time to rework them into something I’m more satisfied with and put them into TeX. Does doing this make me an “active researcher”? What if I get this done and publish the result as a monograph, will that make me an “active researcher”? What if I don’t want people to have to pay for it, so don’t go to a publisher, but just put the thing on my web-site? Will I be an “active researcher” then?

I think the above makes the problems with the “active researcher” standard clear, and the current situation is that I’m claiming I am one, Jacques is saying I’m not even close. In this situation, I am being evaluated by Jacques according to a standard he has chosen. Personally I have no doubt that his evaluation is highly colored by the fact that I disagree with him about string theory. Ever since I first publicly raised criticisms of string theory, he has repeated personally attacked me in various internet forums, for some of the details of this, see this web-page.

This page doesn’t even mention one of the saddest examples of Jacques’s intense desire to not allow links to something I have written. Late last year, Raoul Bott, someone who I was extremely fond of and had a huge influence on me, passed away and I wrote an appreciation of him and about some of my memories of him on my weblog (see here). Jacques was a student a couple years behind me at Dunster House, also knew Bott and also wrote about him on his blog. What he wrote included links to other blogs with mention of Bott, all of which linked to mine, but he specifically refused to put in a link to my homage to Bott, even though it was rather longer, had more scientific content, and was based on more personal experience than the ones he did link to. When I noticed this it became clear to me that his loathing of me and my views knows no bounds whatsoever. I’ve never talked to Jacques personally, but first came to his attention when I started criticizing string theory personally. Ever since that moment he has been completely consistent in his behavior, doing everything he can to paint me as an incompetent crackpot and suppress what I have to say. This latest arXiv story and the claim that I’m not an “active researcher” is just more of the same.

Given this history, and the contentious nature of the debate over string theory, the arXiv should not be allowing Jacques to either be involved in setting the standard that is used to decide whether links to my blog will be suppressed, or be involved in evaluating whether I meet any such standard. He should be well aware of this conflict of interest here, and should long ago have withdrawn himself from the arXiv’s discussions on this issue. What is going on here is not about whether I am an “active researcher”, but about how best to suppress public scientific criticism of string theory, including criticism of the kind of “landscape” studies that make up Jacques’s most recent research.

Posted by: Peter Woit on March 6, 2006 4:10 PM | Permalink | Reply to this

Beastliness

I will leave this comment up, lest I be accused, yet again, of attempting to “censor” Dr. Woit (or of other, even more beastly, acts of omission or commission).

All future comments along these lines will, as I warned above, be swiftly deleted.

Posted by: Jacques Distler on March 6, 2006 5:20 PM | Permalink | PGP Sig | Reply to this

Re: ArXiv Trackback Policy

I think that this discussion gets very childish.
As adult scientists we should immediately stop this.

On the trackback feature:
It should be stopped.
Maybe it could help indentifying interesting papers, but, here’s an example of a blog entry from a person stated as an active researcher:
http://motls.blogspot.com/2006/03/theory-of-everything-from-trinions.html
If I would be smolin, I would recognize at least the picture at the end of this Blog entry as an offense.

Arxiv should be a scientific archive. It should not be the place where persons can outlive their rivalries and conflicts.

Blogs are webpages where one can say anything he like and therefore likely a place for ranting and mocking.

Clearly something Arxiv should not have to do anything with.
Good blogs, including scientific ones, will be seen by many users anyway.

Posted by: Benni on March 6, 2006 5:44 PM | Permalink | Reply to this

Re: ArXiv Trackback Policy

Sorry to say but the “offensive” picture of the “muon neutrino” :-) is taken directly from their paper. I think that you have probably not read that paper yet, and if you did, then you simply don’t understand physics enough to make any judgements about it.

The paper is a complete nonsense and I feel that the potential new authors/commenters who can’t understand why should have no access to arxiv.org whatsoever.

Posted by: Lubos Motl on March 7, 2006 8:35 AM | Permalink | Reply to this

Re: ArXiv Trackback Policy

Motl wrote:
The paper is a complete nonsense and I feel that the potential new authors/commenters who can’t understand why should have no access to arxiv.org whatsoever.

I understand this as an offense. Please stop this. I will omit discussing with persons having such a tone, regardless of their publication list because for me, persons with such a tone are simply crackpots and this will me my last reply on you.

Of course to me, the paper of smolin was also, to be polite, not “making a good sense”. But in my opinion string theory doesn’t make one even. But this are my own thoughts, which I have reasons for and won’t discuss here before publishing it in referred journals.

But that the paper of smolin made “not the best sense to me” does not mean, that a scientific Arxiv should have to do with any personal things. Even if it is a person’s humour.

And the possibiliy to see an own paper in the Arxiv which took some time to write down, gets at least linked with such comments, makes me only thinking not to use Arxiv and preferring the referred journals.

This trackback thingy should be shut down immediately.

But it is not only such “joking”. On many other blogs are personal details, travels, even food of authors. What has a scientific archive to do with such a nonsense? nothing!

Posted by: Benni on March 7, 2006 12:06 PM | Permalink | Reply to this

Re: ArXiv Trackback Policy

as an alternative, I think, Arxiv could make its own blog site, where endorsed physicists are free to discuss papers.

Discussing would mean here: commenting on the scientific content, without any insulting language.

As endorsement, the active researcher status is unusable, because, indeed a researcher may pause his publication activities for 10 years due to write a book or such, which might take so long because it is a very special field (the monstrous moonshine conjecture, the fields medall of Borcherds, for exmaple was worked out in 8 years).

What would be appropriate here might be the same endorsement mechanism as for papers, to have now for comment writers. When such a section would additionally be moderated, one could set as a moderating rule to forbid

overly speculative
insulting
inapropriate comments, where inappropriate is used for comments that have not do do much with the physics involved of the paper or with physics and correct math at all.

What also could be important is a more open process when declining a paper or a comment. But therefore I have not much ideas, since the Arxiv should not discuss with cranks and open the doors for them.

At least Arxiv could
make clear rules for rejection (as statet above)
give via email a reason to the person in question why the rejection was (but only in a single sentence)
And have an address, where others (not the person in question, but maybe the endorser) can complain about the rejection and then start a new evaluation process where the person having rejected is not involved. When there is at maximum one rejection possible for one paper, the mechanism would be

1) more transparent
2) it would get rid off the cranks
3) it would allow more than paper insertion but also a commenting papers section.

Posted by: Benni on March 6, 2006 6:13 PM | Permalink | Reply to this

Re: ArXiv Trackback Policy

I think that having a specific arXiv blog is a good idea, although I think that “blogs” are not necessarily the best way to discuss things, and perhaps a “self-moderated” discussion forum as I proposed above (see my previous comment) would even be better.

W.r. to moderation, I think an “open system” is generally the best, especially if many people are involved, which would be the case since most of the scientific community actually consult the arXiv. A slashdot style system using “ratings”, or other systems based on “readers-reviewing” rather than “administrator-reviewing” (which removes a lot of load on the shoulders of the administrators and in fact generally, in my opinion, produces better results due to the large community involved in the reviewing process – take the example of wikipedia) would be fantastic.

Such ideas, I think, should be studied more carefully, and would in my opinion almost certainly lead to improvements on the current trackback system. I was very happy when the trackback idea was implemented since it gives an opportunity to discuss papers online (in such a way that people know that it’s being discussed), but taking into account the above criticism I think there is place for improvement.

In any case, I think an arXiv-based system would be much better since the scientific community (through the arXiv committee) could create it to fit exactly the needs of the community.

Best,
Vincent

Posted by: Vincent Bouchard on March 6, 2006 6:54 PM | Permalink | Reply to this

Re: ArXiv Trackback Policy

I think that there is no right to be able to place trackbacks on arXiv.org. No reasonable blogger would demand such a right. The very essence of a scientific server is that the contributions can’t be “anything”, and to a lesser extent, it is still true in the case of the trackbacks.

Jacques’ job in the committee is probably not the most pleasant one. If I were in his skin, I would probably surrender several of these cases because the trackbacks don’t seem so infinitely important to me anyway.

On the other hand, I would never, ever complain myself that my trackbacks have been deleted or something like that. Only crackpots produce similar complaints. Crackpots always believe that they are the new Galileo Galileis and whenever the established science rejects their proclamations, their feelings of being Galileo Galileis or messiahs increases even more.

Filtering is an important part of the scientific method. Some people may be very unhappy about this process, and it is more or less guaranteed that people like Peter Woit are programmed to be unhappy more often than others.

Posted by: Lubos Motl on March 6, 2006 6:31 PM | Permalink | Reply to this

Re: ArXiv Trackback Policy

Although I agree that Arxiv should delete these trackbacks and in my opinion open its own “commenting papers” section,
moderator, could you please delete the posting of motl above. The writing about crackpots, messias, and the unhappy peter he is associating with, is complete off topic.

Posted by: Benni on March 6, 2006 6:43 PM | Permalink | Reply to this

Re: ArXiv Trackback Policy

Dear Benni,

could you please kindly provide us with an explanation why you believe that my discussion about unhappy Peter and about crackpots and about messiahs who want to save science by their trackbacks is off-topic during a discussion about unhappy Peter and about the messiahs who want to save science by their trackbacks? So far your comment seems very illogical. Being the most eager eraser apparently does not yet make one a rational judge of difficult issues.

Best wishes
Lubos

Posted by: Lubos Motl on March 6, 2006 7:26 PM | Permalink | Reply to this

Re: ArXiv Trackback Policy

Motl wrote:
Only crackpots produce similar complaints.

Since peter makes such complaints you indirectly calling him a crackpot which is offensive and off topic.

Posted by: Benni on March 7, 2006 12:18 PM | Permalink | Reply to this

Re: ArXiv Trackback Policy

I’m generally disturbed at how these arxiv guidelines institutionalize one perspective on what “active research” is. Grigori Perelman published nothing for the eight years it took him to prove the Poincare conjecture; was he not an “active researcher” at this time?

There are many cultures/modes of scientific research. Some people prefer to publish many papers each year, while others prefer to work for many years until they have brought their ideas to completion before submitting for publication. Its highly disturbing that an arxiv policy would penalize researchers with high standards while rewarding those who break up their work into “minimal publishable units.”

Posted by: PM on March 6, 2006 6:40 PM | Permalink | Reply to this

Re: ArXiv Trackback Policy

The trackback idea itself seems like a good one, as it provides a way to point interested readers to informal discussion about the content and value of a paper. An alternative way to provide such a service would be to maintain a comment facility on the arXiv itself, similar in spirit to the comment section for blog posts, with posting rights being restricted to (perhaps) those who are also allowed to post papers. This latter approach also allows archiving comments in a more permanent fashion than references to external sources (blogs). But regardless of whether trackbacks or internal comments are supported, the same kinds of issues are likely to arise in deciding who can post.

That it was necessary to create a restrictive (and seemingly arbitrary) policy at the outset is puzzling to me. When setting policies, it usually seems best to watch what happens for a little while to see what problems and abuses occur, and then institute selective policies to try to curb those abuses. This seems like the preferable approach in the case at hand for the following reasons.

First, the arXiv is a repository for technical papers, and it requires a certain level of education to be able to read most of them. This in itself creates an immediate (and high) barrier that should effectively screen out most crackpots. Of course it isn’t a perfect screen, since there are also crackpots with PhD’s as well as truly vocal crackpots who will make uninformed comments based on heresay information from someone else, but given the fairly small number of people in those categories who are likely to publicly post comments, it should be quite possible to handle them on a case by case basis.

Second, most readers of a paper are likely to be educated enough to know whether commentary referenced via a trackback is insightful or just drivel or invective. ArXiv users also are likely to remember whose blogs fall into the ‘drivel’ category, and will likely not bother to follow trackbacks to them after a few disappointments. (There could even be a user rating system for a blog’s value, allowing trackbacks to be listed with those from “most valuable” listed first.)

Finally, since it seems like the entire point of the trackback system is to allow readers to elucidate or critique a paper, anyone looking at trackbacks should automatically assume that the referenced post will be less rigorous, and probably less thought out, than the original paper. Those who want more rigor can ignore trackbacks altogether or only look at those blogs they have (from their experience) identified as having an appropriate level of rigor. In a sense, this is a “let the user decide” approach.

There are ways to reduce trackback clutter that don’t depend on an arbitrary “active researcher” standard. Below is one idea. It would be more work initially to introduce than the current list of approved blogs, but it seems more adaptable and robust against arbitrariness than the current approach.

1. Have a blog registration procedure whereby a blog owner could petition the arXiv administrators for rights to post trackbacks. As part of the procedure, the owner of the blog should be clearly identified (no anonymous blog owners), and should include a curriculum vitae that would be stored on the arXiv server and accessible through a separate section of the arXiv. This would allow interested users to decide whether they think the blog owner can be belived.

2. Have uniform requirements for allowing a blog registration request to take effect. Perhaps it could be automatic for anyone would is allowed to post papers.

3. Require all registered blog owners to maintain certain standards if they are to remain registered. For example, the blog owner should be required to moderate comments to their posts, e.g., to disallow comments that aren’t accompanied by a valid email address (not to be posted), as well as deletion of all comments that are personal attacks. The goal would be to keep the signal to noise ratio acceptable.

4. Have a grievance process whereby arXiv users could post a complaint within the “blog registration” section of the arXiv mentioned in (1) above. These complaints would be viewable by any user, and would allow for viewable responses by the blog owner. No anonymous complaints would be allowed, since that would allow the same person to make multiple complaints from different machines as retribution. After some predetermined number of registered complaints, the trackback administrators could perform a quick investigation and either continue or revoke trackback priveleges. With revocation there could be automatic deletion of all past trackbacks to that blog.

The hope is to give enough freedom for the trackback idea to evolve into something truly useful. It is hard to see ahead of time what all the problems will be, so it seems wisest to allow more freedom (within bounds) at the beginning and deal with problems that actually occur, rather than trying to predict abuses at the outsent and then instituting a heavy handed approach to preclude.

Posted by: Marty |Tysanner on March 6, 2006 6:53 PM | Permalink | Reply to this

Re: ArXiv Trackback Policy

Unlike my previous comment, this one is not about my particular case, but a series of questions about the arXiv trackback policy. I’d appreciate it if Jacques or someone else on the arXiv advisory board could provide answers. For some reason, I’ve never had any luck getting answers about arXiv policy by polite e-mails to people associated with the arXiv, but have been informed about arXiv policy through blog comments.

1. I pretty regularly checked the “recent trackbacks” page over the last few months, and my impression was that by far the largest number of postings for which trackbacks to hep-th papers were generated were at the two blogs hosted at golem.ph.utexas.edu. Could some up-to-date data on this be made available so that actual data about where tracbacks are going might be used to inform the discussion here?

2. What is the arXiv policy about responding to polite e-mails from non-“active researchers” inquiring about why their trackbacks have not appeared? Will they get any response to their inquiry? Will such a response include a statement of the relevant arXiv policy?

3. How many papers must a non-“active researcher” post to the arXiv before becoming an “active researcher”? Is this number different if they’re a graduate student than if they have a significant number of publications to their name but haven’t published a paper in a while?

4. If the arXiv decides to change its current “active researcher” policy, based on discussion here or elsewhere, will this be publicly announced? Perhaps somewhere in a blog comment section?

5. Can the arXiv provide a list of “active researchers” currently allowed to have trackbacks to their blog? The “latest trackbacks” page at the arXiv today announces a trackback to superconducting.blogspot.com. This blog is anonymous and it is not clear who its proprietor is. Are trackbacks to anonymous blogs really all right? If not, can the arXiv tell us who runs this blog? E-mail to its proprietor appears to go to the owner of artilect.org, who is Douglas Bard. Is Douglas Bard an “active researcher”? Checking for his publication record by the approved method of doing an arXiv search turns up nothing.

Posted by: Peter Woit on March 6, 2006 7:37 PM | Permalink | Reply to this

superconducting.blogspot.com

Anonymous blogs are generally not allowed. This one, if I am not mistaken, belongs to Hugo de Garis, Associate Professor of Computer Science at Utah State University.

If that’s incorrect, then it probably should not be on the approved list.

Posted by: Jacques Distler on March 6, 2006 8:42 PM | Permalink | PGP Sig | Reply to this

Re: superconducting.blogspot.com

The blog does appear to be anonymous, but it’s plausible it has something to do with de Garis, who in January it seems co-founded “brainarchitect.org” with Douglas Bard, the proprietor of “artilect.org”, where e-mail to the blog goes. The topics of the blog are somewhat different than most of de Garis’s research. Just a guess, but perhaps it is actually a project of one of his graduate students.

Like Bard, de Garis has no arXiv publications, but he has a long publication list. His main web-page mostly advertises his recent book “The Artilect War: Cosmists vs. Terrans, A Bitter Controversy Concerning Whether Humaniyy Should Build Godlike Massively Intelligent Machines” which is published by “ETC Publications”, and which he is promoting on the UFO show “Coast-to-Coast”.

I guess I see why there is a controversy about the quality of arxiv trackbacks under your chosen moderation system.

Posted by: Peter Woit on March 6, 2006 9:44 PM | Permalink | Reply to this

Re: superconducting.blogspot.com

A lot of computer scientists (perhaps most) do not post their papers to the arXivs.

It may well be that the (presumed?) owner of this blog is a nut. In any case, we frown upon anonymous blogs (because of the inherent problem of lack-of-accountability). So it probably should not be on our approved list, if we cannot verify ownership.

While you’re at it, any other blogs we should yank approval from?

Posted by: Jacques Distler on March 6, 2006 10:16 PM | Permalink | PGP Sig | Reply to this

Re: superconducting.blogspot.com

“While you’re at it, any other blogs we should yank approval from?”

Well, there’s the one run by the “active researcher” Rush Limbaugh-style ranter at Harvard who seems to be one of the few people around who agrees with you that suppressing trackbacks to my blog is a good idea…

But, seriously, I’m well aware of how tricky moderation of high-quality internet sources of information is. It’s a continual problem on my blog. There I’ve chosen to err on the side of inclusiveness and let everyone who has something more or less intelligible and on-topic have their say. It’s a continual struggle to keep this under control.

I’m sure the arXiv has the same kind of struggle over postings. In general it has done a good job with this and the community as a whole is grateful. My understanding is that the wisdom of experience there is now embodied in the endorser system. Your argument that an endorser system won’t work for moderating trackback sources just doesn’t hold water. There’s absolutely no reason to believe that it would lead to a lower quality of trackbacks than your current “active researcher” system which obviously has problems of its own.

Posted by: Peter Woit on March 6, 2006 11:08 PM | Permalink | Reply to this

Endorsements

My understanding is that the wisdom of experience there is now embodied in the endorser system.

No, that’s just half of the filtering system. The other half — the man-hour intensive, wish it were unnecessary half — is the moderation system.

We moderate paper submissions. For various practical reason, it’s unfeasible to moderate trackbacks.

The endorsement system, by itself, does not and cannot achieve the desired signal/noise ratio.

And I believe (though, obviously, cannot prove) that it would be even less effective when applied to trackbacks.

In any case, I’m puzzled that you’re so enamoured of the endorsement system (unless it’s for the purely pragmatic reason that you think it’s easy to game).

The endorsement system is rather odd, when you think about it. It resembles nothing more than an old-boy’s Club where, to become a member, you need to be “introduced” by an existing member.

For the purpose for which it was created (controlling who can submit papers to the arXivs), it is probably the optimal solution, given the constraints. But, for all the reasons I’ve stated in my post above, it’s not going to work for the trackback system.

Posted by: Jacques Distler on March 6, 2006 11:27 PM | Permalink | PGP Sig | Reply to this

Re: Endorsements

The endorsement system undoubtedly has its faults, any system does. But commenters here and elsewhere are pointing out to you the problems with your current system. You aren’t using the dictionary definition of “active researcher”, you’re making up your own, and not able to tell us precisely what it is. It’s not getting a lot of support except from Lubos.

Everyone wants a high signal to noise ratio, but the problem here is that different people have different ideas about what is signal and what is noise. I happen to think my blog is pure signal, Lubos’s is 90% noise. He undoubtedly thinks the opposite. Each of us can find a sizable number of respectable members of the community who agree with us.

No, not by “gaming the system”, but because there is legitimately a wide range of opinion out there, much more so than at most points in history. This is because no one has a really compelling idea about how to move forward in particle theory.

What you think is “noise” other people legitimately think is “signal” (and vice-versa…). If you want arXiv trackbacks to carry the full range of interesting “signal” that is out there, you’re going to have to put up with a certain amount of noise.

Your argument that individual trackbacks can’t be moderated, so the endorsement system won’t work just doesn’t have anything to back it up. I understand you can’t moderate individual postings, so you have to evaluate blogs on their overall signal to noise level. But why should a blogger who has one or more endorsers be any more likely to produce a higher noise level than one who fits your personal definition of an “active researcher”? You should be well aware of the noise level in the literature these days. By your “active researcher” criterion, the Bogdanovs a couple years ago could set up a blog and you’d be accepting their pure noise into the trackback system.

Posted by: Peter Woit on March 7, 2006 12:25 AM | Permalink | Reply to this

Re: Endorsements

The endorsement system undoubtedly has its faults, any system does.

I didn’t say it was faulty. I said it was half of a two-step filtering system. One half doesn’t work without the other. Why are you having trouble understanding that concept?

But commenters here and elsewhere are pointing out to you the problems with your current system.

The most prevalent opinions are that we should either

shut down the trackback system entirely or
open it to all and sundry.

Since 2 is a nonstarter, the logical conclusion would be to go with 1 …

I happen to think my blog is pure signal, Lubos’s is 90% noise. He undoubtedly thinks the opposite.

If that’s what you think, then you should suggest a trackback acceptance policy which would exclude Luboš. We certainly don’t want to be accepting trackbacks from blogs that are 90% noise, now do we?

More seriously, there’s a saying in the legal profession that “Hard cases make bad Law.” Judging the trackback system by Luboš’s 56 trackbacks (or, for that matter, Peter Woit’s would-be trackbacks) is likely to result in bad policy.

If you want to make a case for a different policy, point to some blogs (aside from your own) that are excluded under the current policy, but “should” be included. And point to some blogs (other than Luboš’s) that are included under the current policy but “should” be excluded. And then make the case that the new policy would yield a more favourable result than the existing one.

In other words, do something other than special pleading…

If you want arXiv trackbacks to carry the full range of interesting “signal” that is out there, you’re going to have to put up with a certain amount of noise.

The dominant sentiment (among those on the Advisory Board who’ve voiced an opinion) is that the noise level under the current system is already too high.

I would welcome any suggestion for modifying the trackback acceptance policy which would raise the signal/noise ratio.

Lowering it will simply get the whole system shut down.

By your “active researcher” criterion, the Bogdanovs a couple years ago could set up a blog and you’d be accepting their pure noise into the trackback system.

And do you claim that they would have any trouble finding endorsements? If so, then I know for a fact that you are wrong. If you haven’t seen any papers of theirs on the arXivs recently, it’s because of the moderation system …

Posted by: Jacques Distler on March 7, 2006 1:06 AM | Permalink | PGP Sig | Reply to this

Re: Endorsements

You just completely ignore my point that the large problem here is that it is not so obvious what is signal and what is noise. Your current system of using an undefined term as a criterion works very well for you to optimize according to your view of what is signal and what is noise, since you get to decide who is an “active researcher”. An endorsement system would put that power in other people’s hands, some of whom have a very different view of what is signal and what is noise than you do.

I just don’t accept your terms of setting the problem that the question is how to further reduce the noise level. The problem instead is that there is too little real useful signal. I’m not worrying about how to suppress the noise coming from Lubos, he’s only one of a large number of noise sources in the particle theory community at the moment, and at least at times he’s kind of amusing.

If you’ll give me a list of bloggers who discuss hep-th papers who are not “active researchers” and are having their trackbacks rejected, I’ll be happy to tell you which ones I think have a useful signal and could find endorsers. I don’t know at all who is on this list, maybe it’s short, maybe it’s long, maybe it’s all crackpots. I was just over at the blog “Life on the Lattice” of Georg von Hippel, if he’s not an “active researcher”, I guess he’d be an example.

I obviously don’t know enough about the endorsement system to know why you have people endorsing the Bogdanovs. If you do, your moderator is acting as an override of a broken endorsement mechanism, trumping a bad endorser. You could still have the same override mechanism in the trackback case. The problem with the Bogdanovs isn’t that some of their papers are good and some are bad, so they need to be individually moderated.

Posted by: Peter Woit on March 7, 2006 1:45 AM | Permalink | Reply to this

Re: Endorsements

I just don’t accept your terms of setting the problem that the question is how to further reduce the noise level. The problem instead is that there is too little real useful signal.

Again, name a blog that you read (aside from your own) which would contribute to the “signal”, were its trackbacks not blocked.

I was just over at the blog “Life on the Lattice” of Georg von Hippel, if he’s not an “active researcher”, I guess he’d be an example.

Life on the Lattice is on the approved list (by virtue of its previous owner, Matthew Nobes). Georg may just be starting his research career, but he has produced two papers this past year (and a conference proceedings) so it seems reasonable to keep him on the list.

I obviously don’t know enough about the endorsement system…

That about sums it up, dunnit?

Posted by: Jacques Distler on March 7, 2006 2:14 AM | Permalink | PGP Sig | Reply to this

Re: Endorsements

Good to hear we are on the approved list.

My two or three attempts to post trackbacks to the arXiv have failed, though, so I suppose there is some technical trouble either with the trackback system (when I click on the “recent trackbacks” link on the arXiv homepage, I get a “cyclical link” error in my browser [konqueror], btw) or at my end (I use blogger, so I had to try and post the trackbacks manually).

Posted by: Georg on March 7, 2006 8:02 AM | Permalink | Reply to this

Re: Endorsements

“That about sums it up, dunnit?”

Always a pleasure to try and have a serious discussion with you, Jacques.

My mistake last night, because of the lateness of the hour, of getting into a side discussion here, one that has nothing to do with what is really at issue.

What criterion you use to moderate trackbacks is besides the point here. My claim is that, under any moderation system designed to identify blogs with a high signal to noise ratio and block those with a low one, my blog should pass that test. If you want to use an “active researcher” criterion, fine. I am one, as I have argued in detail here. The problem with your “active researcher” criterion is that you have left it ill-defined, and are abusing it because you don’t happen to like my scientific views. I’m not the only one pointing out this problem to you.

There are actually two quite different sorts of “noise” that are at issue here. One is bad, uninformed science, but the other is incivility. I see that Joe Polchinski has written in, and the civility issue is part of the problem he is identifying. Whatever the arXiv policy is, “active researcher”, endorsement, or something else, some mechanism should be found to strongly discourage the kind of incivility that is too prevalent in these blogs. Lubos is by far the worst offender, but, with your continual attacks on my competence, Jacques, you’re not far behind. I’ve undoubtedly made mistakes in the past in responding to this kind of incivility. That’s kind of the problem with allowing anyone to start it, it tends to spread.

Posted by: Peter Woit on March 7, 2006 8:42 AM | Permalink | Reply to this

That’s it …

What criterion you use to moderate trackbacks is besides the point here.

No, it is the only point here. The particulars of your case are beside the point.

I see that Joe Polchinski has written in, and the civility issue is part of the problem he is identifying.

No, he was complaining that “This post clearly has no science content.” and hence that “the arXiv, should [not] be providing pointers to [it].”

Whatever the arXiv policy is, “active researcher”, endorsement, or something else, some mechanism should be found to strongly discourage the kind of incivility that is too prevalent in these blogs.

I agree. If someone could suggest a mechanism for doing that, I’d be all ears.

…but, with your continual attacks on my competence, Jacques, you’re not far behind.

In the 3 1/2 years that I have run this blog, I have mentioned your name a grand total of 4 times. (Which, apparently, is not frequently enough for you.)

So much for my “continual attacks.”

You, by contrast, bring up my name with alarming regularity, (96 times, as of this moment) invariably coupled with some insult or accusation.

I don’t know why you’ve taken such an intense dislike to me. While I can’t do anything about what you say about me elsewhere, I’m not in the mood to tolerate your wild accusations over here.

Since you have repeatedly violated the ground rules that I set out for comments on this post (stick to a discussion of the general policy, not the specifics of your own case), I am, very regretfully, going to ask you to take the rest of your comments elsewhere.

Posted by: Jacques Distler on March 7, 2006 9:39 AM | Permalink | PGP Sig | Reply to this

Re: That’s it …

Before this trackbacks issue arxive was a nice place.
No childish offensive comments, no jokes about papers, no questions what is a scientifically appropriate comment, no questions what an active researcher is
and no questions if a moderator has scientific and reasonable rules for blocking a trackback or if he would personally judge.

This was before the trackback system came. That means:
To shut it down is the best.

BTW:
Why does Arxiv board member ethan write at cosmic variance, that he
“does not want to hazard a guess, wether the blog of woit has been the reason for this “active researcher” criteria”?

Why does he write at cosmic variance
“If you think this is not an appropriate criterion, send a note to the advisory board. Public feedback is good”?

Why does he write
“Peter, I don’t consider you as a crank. If you think you are an active researcher, I would strongly oppose against this decision to the arxive board”?

Why does Ethan do this?
From an outside position, and as an astronomer, ethan has such one, this discussion looks quite strange.

If one would find a mechanism, which looks only at the question: “are these blog entries purely related to science and an appropriate discussion of the literature”, then many entries of peters blog might be allowed.

But there is no such mechanism. So everytime a moderator rejecting a trackback would be in danger of being called a censor from those who like to read the blog in question.

What, if everyone with a phd. and an university affiliation could send trackbacks from blog?
Then we would certainly have trackbacks by nobel laureate Brian Josephson about esoterics linked with Arxiv.

What if we say, only entries that comment papers and are commited to science only are allowed?
Then we would have many authors which are insulted from short hypocritical comments, and there would be flamewars on critics about papers starting out which are not appropriate or wrong.

There is only one solution:
No trackbacks at all in a scientific archive.
It is to much personal information there in Blogs. This can’t be moderated.

Posted by: Benni on March 7, 2006 12:49 PM | Permalink | Reply to this

Read the post Catfight Continued
Weblog: Uncertain Principles
Excerpt: The comment I would post at Jacques Distler's blog, if I were allowed to post comments at Jacques Distler's blog.
Tracked: March 6, 2006 8:11 PM

Re: ArXiv Trackback Policy

I would like to write in support of an arXiv policy that restricts trackbacks, but a different policy than the one that is actually in place. In support of my argument I quote in full one blog post regarding my Scientific American article with Raphael Bousso, `The String Landscape:’

“One article in the magazine doesn’t really have much to do with Einstein and I believe would make him gag if he were still around. The article, entitled “The String Theory Landscape” is by Raphael Bousso and Joe Polchinski. In it they claim credit for the pseudo-scientific idea of “explaining” the value of the cosmological constant by the existence of the “landscape” and the anthropic principle. It’s sad to see this nonsense being purveyed by the most respected and well-known popular science publication in the US.”

This post clearly has no science content. To those who would quickly retort `the landscape has no science content,’ I offer as counterexample the paper `Is There a String Theory Landscape?’ hep-th/0309170 by Banks, Dine, and Gorbatov, which is a set of well reasoned scientific challenges to the landscape idea.

The arXiv is a repository for serious scientific work and discussion. Statements such as `this would make Einstein gag,’ and so on are not a part of a serious scientific discussion and I do not believe that the science literature, i.e. the arXiv, should be providing pointers to them.

However, I am not sure that there is a defensible policy that singles out individual blogs; rather, I am skeptical of the trackback idea. It is true that the `permitted’ blogs do in general discuss the scientific content of the papers that they point to, though not always in the most reasoned fashion. The problem is that blogs are ephemeral. Nobody blogs about last week’s paper; the emphasis is on doing it fast not doing it right. A fraction of posts have some lasting value, but more in the nature of a trivia collection than a scientific literature. I do not see the sense of cluttering the arXiv with a set of permanent pointers to these quick-hit remarks. Writing a paper is a serious undertaking, and any response recognized by the arXiv should be equally serious.

Blogs are fine, and I waste time reading them than I would like to admit, but if trackbacks are to be implemented I believe that this should be in the form of some independently maintained overlay to the arXiv rather than the arXiv itself.

Posted by: Joe Polchinski on March 7, 2006 1:06 AM | Permalink | Reply to this

Old News

Nobody blogs about last week’s paper

Actually, I do, but that’s because I am perpetually behind …

More seriously, I think there’s a fair amount of value clustered near the top of the table that I posted above.

But I seriously wonder whether it is worth a fraction of the time I have wasted on this faux controversy.

Posted by: Jacques Distler on March 7, 2006 1:24 AM | Permalink |

Musings

Skip to the Main Content

March 5, 2006