## March 5, 2006

Ever since the trackback system at the arXivs was announced, it was clear that, sooner or later, a controversy would erupt. And, indeed, one has, surrounding the trackbacks of well-known 'Net personality, Peter Woit.

From the beginning, it was made clear that trackback privileges would not be open to all and sundry. The arXivs are a vehicle for communication between research scientists. Not everyone can have their papers appear on the arXivs. Similarly, not everyone would be able to have their trackbacks appear.

Woit has loudly protested the decision not to accept his trackbacks, and the discussion has spread elsewhere in the blogosphere. I have refrained, up till now, from commenting publicly because as a member of the arXiv Physics Advisory Board, I feel very constrained about what I can say publicly, either about the specifics of the case at hand, or about the internal deliberations of the Advisory Board.

But one thing became clear in the discussion over at Cosmic Variance. There’s a lot of confusion about the trackback policy. Some of that confusion was deliberately sown by people with an axe to grind; some of it was the unfortunate result of the less than transparent process under which the policy was developed.

So, what I’m going to do is try to explain the thinking that went into the policy, and then solicit your feedback.

Before plunging into details of policy, we ought to ask what the results of that policy have looked like. Overall, in its few months of operation, the system has worked quite well. There are several dozen blogs/websites from which we currently accept trackbacks. Collectively, they’ve generated several thousand trackbacks. The websites span the full range of fields represented on the arXivs, from high energy theory to condensed matter physics to quantum computation to astrophysics to computer science to quantitative biology. In high energy theory (in which I’ll include hep-th, hep-ph, gr-qc and hep-lat), there are about a dozen websites. In terms of trackbacks generated, the most prolific among them is John Baez’s This Week’s Finds, which has generated more trackbacks than all the other high energy theory sites combined. Trailing rather badly behind are this site, The String Coffee Table and all the rest. Overall, the most prolific site is NetAdv, which features review articles on a wide variety of fields of physics. It has generated more trackbacks (nearly 2400) than all the other sites, in all fields, combined.

Those sites with the largest number of trackbacks are, of course, those which have been around the longest. They may not retain their leads, going forward. But they are indicative both of the diversity and of the generally high quality of the trackbacks in the system.

Before talking about the trackback policy, we should start with the acceptance policy for papers. For a paper to be accepted to appear on the arXivs, it must go through a two-stage filter.

1. The author must be an approved submitter, usually through having been endorsed.
2. Each paper from an approved submitter must be accepted by the moderator for that section of the arXiv.

For a variety of practical reasons, it was decided that it would be infeasible to moderate each individual trackback. Trackbacks would go through just a single stage of filtering. If the author (or, more precisely, the author’s website) is approved, then (all of) his trackbacks may appear.

But what should the criteria for acceptance be?

One’s first thought is: why not use the same endorsement mechanism used for paper submission? Unfortunately, the experience of the moderation system is that endorsement is not a terribly high barrier to entry. Some endorsers are rather loose in endorsing people to submit papers and one can only imagine that they would be even looser in endorsing people to submit trackbacks. In the case of papers, the second-stage filter of moderation is clearly necessary. But we had already decided that there would be no such second stage in the case of trackbacks.

It is also vital to have a reasonably objective standard. “This looks like an interesting weblog.” was not going to be a workable criterion. Nor would any number of other subjective criteria.

The solution which was adopted, in the end, was that trackbacks would be accepted if they come from active researchers. It’s not particularly hard to figure out who’s an active researcher: just look at their publications. Exactly what level of activity counts as “active” is an issue. Wherever you draw the line, there will be borderline cases that require a judgement-call. But in most cases, the decision should be (and, indeed, has proven to be) straightforward.

The “active researcher” criterion is not perfect. A young graduate student, for instance, may not yet have much of a publication record but might, nonetheless, have useful things to say.

The decision whether to accept trackbacks from a given site is not immutable. In the case of a graduate student who started blogging well before she started publishing, it’s easy to start accepting her trackbacks once she has a few publications and has established herself as a researcher.

Conversely, nothing prevents us from stopping accepting trackbacks from previously acceptable sites if the author is posting defamatory, plagiarized, or otherwise unsuitable material. The same holds true for the ability to submit papers to the archives.

But, in the main, since trackbacks are an adjunct (a useful adjunct, but still an adjunct) to published papers, it’s not unreasonable to restrict trackback privileges to those who are actively engaged in research, as evidenced by their published papers1.

That’s the trackback policy, as it’s currently constituted. Peter Woit’s publication record doesn’t put him anywhere close to “active researcher” status. So, without some radical change in our criteria (or a radical change in his level of research activity), his trackbacks are not accepted.

Generally, there has been satisfaction with the trackback system, but some on the Board have expressed the feeling that the existing trackback acceptance policy is too lax. The scientific quality of commentary in the blogosphere is far too low, and we should either tighten our acceptance criteria or shut the whole system off.

I don’t know that I see a good way to tighten our acceptance criteria, so if it becomes the consensus that the trackbacks accepted under the current system are of generally too low a quality, it may indeed be that the only solution is to simply shut the system off.

I’d now like to turn this over to you, the readers, for discussion. I would like to hear suggestions for improving the trackback acceptance policy, or the trackback system at the arXivs generally. Or perhaps you agree with those on the Board who think this whole blogosphere thing is a waste of electrons and the trackback system should simply be shut down.

But first, some ground rules.

1. Since I’m constrained from talking about the specifics of the Woit case (beyond what I have said above), so are you. There will be no pleading of Peter’s case here. Nor will attacks on Peter be tolerated. In both cases, such comments will be swiftly deleted. We will restrict the discussion to matters of general policy. And there’s very little about the specifics of this particular case that need to or ought to appear in such a discussion.
2. If you want to propose an alternative trackback acceptance policy, remember the design criteria:
• It needs to be website-wide; we’re not going to start moderating individual trackbacks.
• It needs to be reasonably objective. Subjective criteria will just lead to endless controversies.
• It needs to be at least as restrictive as (maybe more restrictive than, if you want to satisfy the trackback-sceptics) the current policy.
3. It’s probably not worthwhile trying to pin down what the precise boundary between “active researcher” and John Q. Blogger should be. As I said, wherever the precise line is drawn, there will always be borderline cases. But with any reasonable choice of where to draw the line, Woit isn’t one of those borderline cases.
4. Finally, we are not going to turn this into a discussion of “censorship” by the arXivs. If you are banned from posting papers to the arXivs, you may have a legitimate grievance. But this is not the place to air such grievances. Again, such comments will be deleted.

So that’s it. I’ll don my asbestos underpants and let y’all have at it …

#### Update:

Paul Ginsparg has sent me the latest stats from the trackback database. There are currently 5132 trackbacks from 51 approved sites. In high energy theory, the sites with 30 or more trackbacks are:
High Energy Theory Sites with 30 or More trackbacks at arXiv.org
This Week’s Finds731
The String Coffee Table300
Musings268
Luboš Motl56
Cosmic Variance33
Also, for those commenters advocating an arXiv-sponsored blog, an arXiv-sponsored slashdot-style site, an arXiv-sponsored comment forum, …it’s not gonna happen. Please take the time to read my previous post on the rationale behind the trackback system, if you want to understand why.

1 It probably goes without saying that you couldn’t use the same criterion to decide who can submit papers to the arXivs. That would be a Catch-22.

Posted by distler at March 5, 2006 11:38 PM

TrackBack URL for this Entry:   http://golem.ph.utexas.edu/cgi-bin/MT-3.0/dxy-tb.fcgi/760

It seems to me that the crucial question here is not “Is the commenter an active researcher?” but rather “Does the site offer technical discussion of the subject matter?”.

“Active researcher” seems very difficult to implement in practice. You end up having to make some rather arbitrary definitions, which are then immediately supplemented by exceptions for retired faculty, young grad students, etc,… What if Matt Nobes starts blogging again? What if Grothendieck decides he wants to speak up?

“Technical discussion”, on the other hand, seems fairly easy to implement. Are there equations? Error bars? Is the language appropriately high level? Does the commenter consider the specific details discussed in the paper, or is the commenter offering only generalities?

And as an added bonus, this sort of policy would encourage people to raise their standards for scientific discourse, rather than igniting flamewars about credentials.

–A.J.

ps. I hope the underpants hold up.

Posted by: A.J. on March 6, 2006 2:16 AM | Permalink | Reply to this

“Technical discussion”, on the other hand, seems fairly easy to implement. Are there equations? Error bars? Is the language appropriately high level? Does the commenter consider the specific details discussed in the paper, or is the commenter offering only generalities?

To the contrary, by virtue of being inherently a judgement-call, it’s far harder to implement in a consistent and impartial manner.

Indeed, the qualities you would use as the basis for deciding whether to approve a site’s trackbacks vary, in my experience, rather dramatically from post to post on some sites. Some posts may be full of technical details; others might be rather cursory.

Unless we’re willing to moderate individual trackbacks, it’s hard to base an approval system on the content of individual posts.

And as an added bonus, this sort of policy would encourage people to raise their standards for scientific discourse, rather than igniting flamewars about credentials.

Raising the standards of scientific discourse in the blogosphere is certainly an laudable goal (we, here at Musings, strive for the highest standards of scientific discourse).

Flame wars are one of, but far from the only, thing dragging down those levels. But if those flame wars don’t generate trackbacks to the arXivs, then they are irrelevant to the discussion at hand.

On the other hand, as I said, there are those who argue that level of scientific discourse, probably even of the posts here at Musings, is too low for the arXivs to want to be associated with them — even at the arms-length afforded by the trackback system.

Posted by: Jacques Distler on March 6, 2006 2:57 AM | Permalink | PGP Sig | Reply to this

To the contrary, by virtue of being inherently a judgement-call, it’s far harder to implement in a consistent and impartial manner.

Yes, of course it’s a judgement call. So is “active researcher”; that’s the basis of all these complaints.

But asking for technical commentary keeps out the crackpots while still allowing a wide variety of commentary. Ask for more than that, and you have three choices: 1) make arbitrary distinctions, 2) make editorial decisions case by case, or 3) raise your standards very very high.

Personally, I can see the case for 3). And honestly, it’s not that hard to distinguish between the sort of technical commentary offered at String Coffee Table and Life on the Lattice, and the more general sort of commentary you find at Cosmic Variance or Not Even Wrong.

But if you want 1) or 2), you really must clarify: What are you asking for, beyond banning crackpots?

And out of curiosity, what have the current standards actually accomplished? Who is banned? Who is not?

Posted by: A.J. on March 6, 2006 2:28 PM | Permalink | Reply to this

“Technical discussion”, on the other hand, seems fairly easy to implement.

I disagree. Peter’s case is instructive here. The spectrum of publically aired opinion on this runs from “anti-science crackpot”, through “the same critique we’ve heard for years”, to “wrong, but worth listening too” and “a sharp critique, exposing major flaws in HEP” (those aren’t acutal quotes, just my interpretation of the opinions I’ve read). All those seem to be opinions held by “active” physicists. So how does one decide “technical” or not?

That being said, I think the active researcher standard is off the mark as well. The “active researcher” idea totally misses the possibility of an outsider, who may have a valid point, being seen by those who read arxiv trackbacks. Instead, this standard will only be physicists commenting on other physicists.

What if Matt Nobes starts blogging again?

Not likely :), but fortunately I’ve turned my blog over to an active researcher, to carry it forward.

Posted by: Matthew on March 6, 2006 3:09 AM | Permalink | Reply to this

I agree more or less with A.J. I think that the goal of the moderation should be to eliminate trackbacks to blogs that don’t discuss the paper seriously or make abusive comments.

To implement this efficiently, arXiv should use a complaint policy. Only the obvious trackbacks to spam sites (e.g. viagra advertisements) should be automatically blocked.

Now suppose that there exists blog that unfairly criticises papers, uses abusive language toward the authors, blocks comments from people defending the paper or does something else distasteful, then people should complain to arXiv. This will probably only happen in a very small prcentage of the trackbacks. Perhaps you’ll have one case per year. Then if the complaint is judged to be valid, trackbacks to the blog will be banned.

Let me give some motivations why you want to have a liberal policy regarding trackbacks and not just allow trackbacks from blogs of people who are experts in the field.

1) What matters is the discussion on the blog and if it is managed properly. If I have a blog and post about an interesting article on string theory, any expert is able to participate in the discussions. The level of the discussion doesn’t necessarily correlate with my knowledge about the subject.

2) Most scientists would like it if there preprints had some trackbacks to blogs managed and read by non experts.

Posted by: Count blis on March 6, 2006 11:17 AM | Permalink | Reply to this

I agree more or less with A.J

We aren’t in complete agreement, however: I lean towards frankly elitist standards, and you seem willing to let everyone have a say.

And “one a year” seems very very low. Just ask any of the sci.physics.research moderators how much time it takes to deal with the crackpots.

Posted by: A.J. on March 6, 2006 2:36 PM | Permalink | Reply to this

A.J., I don’t think you would have many cranks posting trackbacks to arXiv articles. Usenet groups and message boards do attract cranks because they want to let the world know about their new ”theory of everything”. They usually aren’t interested in real physics articles at all.

Another thing: Perhaps one should allow the author of an arXiv preprint to block trackbacks to his preprint for whatever reason. So, Lubos Motl would probably remove trackbacks from Peter Woit’s blog to his article, but I would decide to keep them :)

Posted by: Count Iblis on March 6, 2006 4:35 PM | Permalink | Reply to this

Thank you for bringing this up. This seems a bit of a tempest in a teacup…but an important one. Effectively, as you explain it, the arxiv would like to create an ‘in-group’ and an ‘out-group’ to keep discussion both serious and scientific. One proposed division is that ‘active researchers’ be the considered the in-group.

Not a bad start. The aim is noble. The reasoning inspired. The motive, undoubtedly pure, public spirited, and unsullied by rent-seeking behavior. But let’s consider at least a few harmless thought experiments to test any exclusionary policy:

Would Watson or Crick have been active researchers to Erwin Chargaff before their big break? Would Alexander Grothendiek be allowed trackbacks as a blogger if he started his own site in his current state? How about Irving Segal as a cosmologist (rather than a mathematician)? Would John Nash be allowed to comment on game theory when not in a proper frame of mind? What about Papakyriakopoulos before Ralph Fox gave him legitimacy at Princeton? How about letting Teichmuller and Jordan share their views on national socialism (as well as moduli and commutative algebras). What if Streleski or Kaczynski wishes to start picking up research (and blogging) again? The Bogdanovs? Luca Turin? Kary Mullis? Ramanujan (pre Hardy)?

The trouble for your goup seeking to put up a velvet rope is that there are a disproportionate number of fascinating minds concentrated among the truly marginal cases. It’s like counting the proportion of stellar individuals among the tiny group of Harvard dropouts: it is, gloriously, somewhat higher than might otherwise be expected.

Alas, science isn’t a guild run garden party with a guest list. At its best, it’s more like a free-for-all rave where we all throw up our ideas to see which brainwaves stick to the walls.

Like it or not, the next big thing can come from anywhere and, more importantly, anyone. Creeps, cranks, gadflys, losers, felons, anti-semites, lunatics and [perhaps worst of all] non-tenure track faculty (horrors!) have all contributed to the edifice of science.

You will eventually face this problem, but the blog that started this issue rolling is an IDIOTIC test case. You have no idea how many top flight active researchers tune in to the blog that is motivating your post. Please, stop protecting us from such dangerous people. In the case of Dr. Woit, it makes it look like String theory dost protest too much.

Look. Science isn’t always pretty. Make a policy, announce it and stick to it with minimal squishiness. The worst thing you can do is resort to backhanded dismisals of people who are not ‘active researchers’.

Requiring a reputible PhD will solve most (though not all) of the problem. Then, just hope against hope that the next big idea comes from one of an A-list colleague and not a disgruntled outcast.

A Friend

Posted by: c niedman on March 6, 2006 3:10 AM | Permalink | Reply to this
Weblog: Uncertain Principles
Excerpt: There's a kerfuffle in the physics blogosphere these days over the somewhat arcane issue of TrackBacks to posts on the ArXiV, the commual preprint server where researchers can post drafts of the papers that they have submitted to research journals...
Tracked: March 6, 2006 7:16 AM

To me it would already be helpful if somewhere there were a list of “allowed sites”. Not only because that would give one more ways to spend one’s idle time but it would also tell you if you are facing technical problems or you are not whitelisted.

Me for example, was in the past able to post trackbacks via atdotde (at least that was my impression, viz e.g. http://www.arxiv.org/abs/astro-ph/0509039 where the “trackback” link currently gives me a “redirection limit exceeded” error). But my most recent atempt failed: When entering the ping into Haloscan for http://arxiv.org/abs/hep-th/0602072 there was no error message but the trackback has not appeared up to now.

So, do I have problems with haloscan? Am I not active enough to post trackbacks (both paper- and blog-wise)? I don’t know. If there were I list of trackback enabled sites, I could tell.

Robert

Posted by: Robert on March 6, 2006 7:16 AM | Permalink | Reply to this

http://www.arxiv.org/abs/astro-ph/0509039 where the “trackback” link currently gives me a “redirection limit exceeded” error

One of the many bugs in the system.

To me it would already be helpful if somewhere there were a list of “allowed sites”.

Right now, as far as I understand, the only way to find that out is to have direct SQL-query access to the database. (The statistics I cited above are 2-month old statistics furnished by Paul Ginsparg. adotde was on the approved list.)

Add this to the TO-DO list.

Posted by: Jacques Distler on March 6, 2006 7:55 AM | Permalink | PGP Sig | Reply to this

Is the list of “allowed sites” available, or is that still a TO-DO item? I’d wanted to note on EUREKA the sites for which trackbacks are allowed, but I wasn’t able to find that information at arXiv.org or elsewhere.

Posted by: Blake Stacey on December 10, 2007 3:39 PM | Permalink | Reply to this

Is the list of “allowed sites” available

Not to my knowledge. The only web interface I know about is this one.

or is that still a TO-DO item?

I have no idea. I am not at all involved in the implementation, the management or the operation of the Trackback Service.

Posted by: Jacques Distler on December 10, 2007 3:56 PM | Permalink | PGP Sig | Reply to this

It is clear to me that the scientific level of discussions on the web is not quite there yet, but there is nothing inherent in the medium that prvents this from happening. I think the trackback system can be one step towards increasing the level of discussion and encouraging more active researchers to participate and offer their points of view. When this happens, and it does occasionally, one can get a glimpse of something indispensable. Hope then that trackbacks continue, and the current level of moderation looks good to me.

Posted by: Moshe on March 6, 2006 8:34 AM | Permalink | Reply to this

——-BEGIN SIGNED MESSAGE——- Hash: SHA1

Ah… I had written a long comment, but my browser crashed on me on i hit previewsigh

Anyway, all i mean to say is that: The way it’s currently set, the system uses “paper count” to classify allowance into the trackback system. What i think is that you’ll need a more plural and diverse approach to this problem (given its very nature and setting – the ‘Net). Right now the system allows those folks that have a high ‘paper count’ with very little to say. OTOH, those folks with a lower ‘paper count’ but with more ideas to add and contribute to a given discussion, are left out.

I do realize that this 2nd category is the one that lies mostly on the “boundary” of the 1st one, but i think that an equally arbitrary set of rules can be used, such as: institution filiation (reputable university/college?), academic heritage (reputable advisor? postdocs?), allowed to publish on the arXivs, etc. (Note that these rules can be extended for different countries.)

I’m not claiming those to be a ‘good’ set of rules but it’s my opinion that they are as good as ‘paper count’. (Sure enough, my opinion may not be of great value… ;)

Also, although i’m not suggesting manual moderation, i don’t see why the following scenario would be ‘bad’: A given arXivs moderator is surfing the web on s/he bumps into an “interesting” website – it can be submitted to the criteria above and, eventually, added to the list of allowed trackbacks. (This is not to say that people should email requests to the moderators but, it’d be something like “Stumble Upon” in Mozilla. ;)

On a different token: “Thanks, Jacques, for making this subject available to public discussion and heling take care of the arXivs trackback system.” :) ——-BEGIN SIGNATURE——- Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFEBdVTwZPTkJ/pv0sRApSxAKClLQJ7elbAA52WTZiVT6za/7cz4wCcDho+ ycdop5Uhe3pFk/c4xzUFt7M= =u/2s ——-END SIGNATURE——-

Posted by: Daniel Doro Ferrante on March 6, 2006 9:42 AM | Permalink | Reply to this

One way to compromise would be to allow trackbacks for any blog that, say, 20 active researchers (by the current definition) endorse as one they read/comment on frequently.

Posted by: Tez on March 6, 2006 10:29 AM | Permalink | Reply to this

Dear Jacques,

I agree with those on the Board who advocate shutting down the trackback system.

At least in the high energy physics community, the arxiv is the primary tool for communcation among scientific researchers. And the central goal of the arxiv should be to maintain as high a level of communication among scientific researchers as possible. Frankly, most of the physics discussion on blogs is simply not at a high enough level to be useful for a researcher. (Your blog is one of the very few exceptions to this statement). Blogs are occasionally useful for providing pointers to papers I’ve missed or summaries of papers I haven’t read, but the vast majority of blog posts are simply not research-grade material. They are typically either too vague, personally biased, irrelevant or incorrect to be useful. This is not to say that blogs are worthless, but only to say that when I am in the mood to wallow in the muck of the blogosphere I am perfectly capable of doing so on my own without the help of an arxiv trackback system. Including trackbacks lowers the overall level of discourse on the arxiv, with very little benefit.

One last comment: the arxiv is neither a tool for communication to the general public nor an informal online discussion forum. Of course, these things are important and useful, but we should maintain a clear distinction between the arxiv as a tool for formal research-level communication, and the blogosphere, which is by its nature informal. If someone would like to set up a physics blog aggregator or something like that, this may be worthwhile. But adding blog trackbacks to the arxiv needlessly blurs the line between real research and informal discussion.

Thanks.

Posted by: Anonymous on March 6, 2006 1:00 PM | Permalink | Reply to this

Frankly, most of the physics discussion on blogs is simply not at a high enough level to be useful for a researcher. (Your blog is one of the very few exceptions to this statement).

This brings up a point that I probably should have emphasized. If you’re looking at this from the perspective of blogs (particularly those which happen to be the most popular and best-known in the blogosphere), you’re looking at it through the wrong end of the telescope.

What’s important here is the actual trackbacks at the arXivs. In our field (high energy theory), the vast majority of trackbacks come from John Baez (way ahead), followed by this blog and the String Coffee Table.

I’d call those three “high quality” sources. Wouldn’t you? All the other blogs combined produce only a miniscule fraction of the trackbacks in high energy theory.

They may be more entertaining to read, they may be vastly more popular, but they are mostly irrelevant, if you are trying to judge the “average quality” of trackbacks at the arXivs.

[That said, I’m not sure that the trackbacks from NetAdv actually add much value. But that’s another discussion…]

Posted by: Jacques Distler on March 6, 2006 1:31 PM | Permalink | PGP Sig | Reply to this

From the beginning, it was made clear that trackback privileges would not be open to all and sundry. The arXivs are a vehicle for communication between research scientists. Not everyone can have their papers appear on the arXivs. Similarly, not everyone would be able to have their trackbacks appear.

I’m not sure I entirely follow the basic premise of disallowing trackbacks from arbitary users. It’s pretty clear that limiting the material uploaded to on-topic items - i.e. scientific research papers - is necessary to make the arXiv useful - otherwise there would simply be too low a signal-to-noise ratio for anyone to bother using the service. On the other hand I’m not quite sure that the same thing applies to trackbacks. The primary use of a trackback (it seems to me) is to point people who are already interested in a paper toward discussion which includes some mention of the paper*. That discussion can be interesting even if it isn’t among professional scientists - for example I guess many people would be interested to know if their paper had been featured on Slashdot (assuming Slashdot sent trackbacks which, sadly, AFAIK, they do not). Indeed, I would say that enabling trackbacks in this way could be a very useful tool in gauging the public reaction to certian pieces of research.

I assume the objection to such a setup is that it would create a signal-to-noise problem? I tend to disagree - I think the number of trackbacks will be small whoever is allowed to contribute, but I’ll note that limiting the field to “active” researchers provides no guarantee of quality - after all there is a time-honored tradition of Nobel Prize Winners Saying Extremely Stupid Things About Fields Outside Their Area Of Expertise. There is nop reason to assume that won’t also apply to the discussion of arXiv papers in another (sub-)field by an “active” researcher.

Of course there’s no reason that all trackbacks have to be equal. One could, for example order trackbacks so that “active” researchers appear at the top of the list and everyone else is relegated to the bottom. I know physicists are generally bad at solving UI problems (at least, the ones who design astronomy software certainly are) but it doesn’t seem too hard to do well enough that it keeps the signal to noise low for everyone who wants to ignore the “little people”.

*Under this definition, there is no particular value in a trackback from a site which simply collates lists of papers. I suppose the current policy makes more sense to me if we assume that you’re viewing a trackback like a traditional citation. However, given the nature of the web I don’t think it makes sense to compare the two.

Posted by: jgraham on March 6, 2006 1:22 PM | Permalink | Reply to this

but it doesn’t seem too hard to do well enough that it keeps the signal to noise low for everyone who wants to ignore the “little people”

but it doesn’t seem too hard to do well enough that it keeps the signal to noise high for everyone who wants to ignore the “little people”

Posted by: jgraham on March 6, 2006 1:25 PM | Permalink | Reply to this

Hi,

I would just like to throw an idea out here, that could perhaps resolve the blog/trackbacks issue.

Instead of having trackbacks to blogs, which are led by only a few researchers (who like and take time to blog), perhaps there could be directly on arXiv a “slashdot” (http://www.slashdot.org) style discussion forum attached to each paper published on the arXiv. Any researcher endorsed to publish on the arXiv could post comments to articles.

Moreover, instead of having to create some sort of “censorship rules” as it is necessary to do with the actual trackback system (whose blog should be endorsed, whose blog should not be), there could be again a “slashdot”-style system of “voting”, where readers can give a “rate” to the comments, according to their pertinence. Then each reader can set its own level of reading, that is for instance “I want to see only comments that have at least 8/10”, “I do not want to see any comments”, “I want to see all comments”, etc.

This kind of system introduces a self-sustainable “censorhip” system that does not (in principle) require the intervention of the administrators of arXiv. The whole research community, through the “rating” system, administrates the discussion forum.

This is even more true if only endorsed people (from arXiv) can post comments on the discussion forums. This sort of system is used on many websites at the moments, not only slashdot; websites that tend towards “open-publishing” but want to keep a certain level of pertinence.

I don’t know what people think about this idea, but I personally think it would be much easier to manage than the trackback system, and also it would give the opportunity to everyone in the scientific community (not only bloggers) to comment on published articles. That could lead to very interesting and stimulating discussions. That would definitely be a “new” way of doing research, but I think we should take advantage of the new possibilities opened up by the internet and our wonderful free publishing system which is the arXiv. Technically I don’t think it would be very difficult to implement, as it is already implemented on many websites (although I do not know how the arXiv works technically).

Best,
Vincent

Posted by: Vincent Bouchard on March 6, 2006 3:17 PM | Permalink | Reply to this

I am very much in favour of establishing new (likely web enabled) possibililties to give feedback to papers in addition to writing a new paper or a “comment on…” or just talking to the people who happen to be around when you fill your coffee mug in the morning (assuming European style where the new preprints arrive in the morning). The simplest version would be to make the names of refereees public who accept a paper. At least today (who knows where we’re heading) blogs are closer to the “ranting to office mates” end of the scale but trackbacks makes them more valuable. It would be bad if the trackback feature would be shut down because of this issue.

However general discussion boards for papers already exist and they don’t work too well as there is far too much noise. Blogs on the other hand provide a slightly more formal frame and at least in theory the author(s) have some credebility to lose.

Posted by: Robert on March 6, 2006 3:57 PM | Permalink | Reply to this

Jacques has asked that the specifics of my case not be discussed here, but then has gone on to announce that

“Peter Woit’s publication record doesn’t put him anywhere close to “active researcher status”.

For “publication record”, he embedded a link not to, well, my publication record, but to the list of my arXiv postings. He is well aware of how SPIRES works and knows that something much more accurate would be this:

publication record

Sure, most of these publications are from the 80s, when I was actively involved in work on a rather hot topic in lattice gauge theory. During the 90s I continued to be actively involved in research, but changed to thinking about quite different things, working on very unpopular ideas that no one else seemed to be working on and thus likely to scoop me. I won’t argue against anybody who tells me that I should have written more of this up as I went along, but I will strongly object to anyone who claims that those years were not spent doing “active research”.

The fruits of this were written up in one moderately long 56 page paper and posted on the arXiv in June 2002 as hep-th/0206135. This was never submitted for peer review, since I didn’t much see the point. I have no doubt I could have found a journal to accept it, but don’t see what that would have accomplished.

This history according to Jacques clearly doesn’t make me an “active researcher”, but he gives no indication why this is the case. If I had submitted the paper for peer review and it had been accepted would I now be an “active researcher”? If I submit it somewhere tomorrow and it is accepted will I all of a sudden become an “active researcher”? Is the problem that it is only one paper? What if I had broken it up into three papers and posted those on the arXiv, would I now be an “active researcher”? What if these three papers were posted over a period of years, what if they were peer-reviewed, etc?

Since 2002 I’ve continued to work actively in this area, and have given talks about this work; slides for talks at a conference in 2003 and at a Dartmouth colloquium in 2004 are available at my web-site. Do those materials count as evidence that I’m an “active researcher”? I have a couple manuscripts in preparation that I’m not yet happy with, but hope to finish and post on the arXiv later this year. If I do this will I become an “active researcher”? Can I put all this together in one long paper, or to become an “active researcher”, will I have to break things up into multiple pieces? How many pieces should I break my manuscript into in order to become an “active researcher”? Will they need to be peer reviewed?

During 2003-2005 I taught various graduate courses here in the Columbia math department, and this experience involved a great deal of involvement with research level math and physics. I put a huge amount of time into writing up graduate course notes that explain the geometric approach to representation theory and how this is related to geometric quantization and physics. Much of this I think is rather original, at least in presentation. It’s available on my web-site, judge for yourself. Does doing this make me an “active researcher”, even though it’s not on my “publication record”? I also spent a lot of time working on a graduate quantum field theory course for mathematicians. Again this involved a sizable amount of research level work, since much of this was original in one way or another. I now have a hand-written notebook filled with these notes, waiting for me to find time to rework them into something I’m more satisfied with and put them into TeX. Does doing this make me an “active researcher”? What if I get this done and publish the result as a monograph, will that make me an “active researcher”? What if I don’t want people to have to pay for it, so don’t go to a publisher, but just put the thing on my web-site? Will I be an “active researcher” then?

I think the above makes the problems with the “active researcher” standard clear, and the current situation is that I’m claiming I am one, Jacques is saying I’m not even close. In this situation, I am being evaluated by Jacques according to a standard he has chosen. Personally I have no doubt that his evaluation is highly colored by the fact that I disagree with him about string theory. Ever since I first publicly raised criticisms of string theory, he has repeated personally attacked me in various internet forums, for some of the details of this, see this web-page.

Given this history, and the contentious nature of the debate over string theory, the arXiv should not be allowing Jacques to either be involved in setting the standard that is used to decide whether links to my blog will be suppressed, or be involved in evaluating whether I meet any such standard. He should be well aware of this conflict of interest here, and should long ago have withdrawn himself from the arXiv’s discussions on this issue. What is going on here is not about whether I am an “active researcher”, but about how best to suppress public scientific criticism of string theory, including criticism of the kind of “landscape” studies that make up Jacques’s most recent research.

Posted by: Peter Woit on March 6, 2006 4:10 PM | Permalink | Reply to this

### Beastliness

I will leave this comment up, lest I be accused, yet again, of attempting to “censor” Dr. Woit (or of other, even more beastly, acts of omission or commission).

All future comments along these lines will, as I warned above, be swiftly deleted.

Posted by: Jacques Distler on March 6, 2006 5:20 PM | Permalink | PGP Sig | Reply to this

I think that this discussion gets very childish.
As adult scientists we should immediately stop this.

It should be stopped.
Maybe it could help indentifying interesting papers, but, here’s an example of a blog entry from a person stated as an active researcher:
http://motls.blogspot.com/2006/03/theory-of-everything-from-trinions.html
If I would be smolin, I would recognize at least the picture at the end of this Blog entry as an offense.

Arxiv should be a scientific archive. It should not be the place where persons can outlive their rivalries and conflicts.

Blogs are webpages where one can say anything he like and therefore likely a place for ranting and mocking.

Clearly something Arxiv should not have to do anything with.
Good blogs, including scientific ones, will be seen by many users anyway.

Posted by: Benni on March 6, 2006 5:44 PM | Permalink | Reply to this

Sorry to say but the “offensive” picture of the “muon neutrino” :-) is taken directly from their paper. I think that you have probably not read that paper yet, and if you did, then you simply don’t understand physics enough to make any judgements about it.

The paper is a complete nonsense and I feel that the potential new authors/commenters who can’t understand why should have no access to arxiv.org whatsoever.

Posted by: Lubos Motl on March 7, 2006 8:35 AM | Permalink | Reply to this

Motl wrote:
The paper is a complete nonsense and I feel that the potential new authors/commenters who can’t understand why should have no access to arxiv.org whatsoever.

I understand this as an offense. Please stop this. I will omit discussing with persons having such a tone, regardless of their publication list because for me, persons with such a tone are simply crackpots and this will me my last reply on you.

Of course to me, the paper of smolin was also, to be polite, not “making a good sense”. But in my opinion string theory doesn’t make one even. But this are my own thoughts, which I have reasons for and won’t discuss here before publishing it in referred journals.

But that the paper of smolin made “not the best sense to me” does not mean, that a scientific Arxiv should have to do with any personal things. Even if it is a person’s humour.

And the possibiliy to see an own paper in the Arxiv which took some time to write down, gets at least linked with such comments, makes me only thinking not to use Arxiv and preferring the referred journals.

This trackback thingy should be shut down immediately.

But it is not only such “joking”. On many other blogs are personal details, travels, even food of authors. What has a scientific archive to do with such a nonsense? nothing!

Posted by: Benni on March 7, 2006 12:06 PM | Permalink | Reply to this

as an alternative, I think, Arxiv could make its own blog site, where endorsed physicists are free to discuss papers.

Discussing would mean here: commenting on the scientific content, without any insulting language.

As endorsement, the active researcher status is unusable, because, indeed a researcher may pause his publication activities for 10 years due to write a book or such, which might take so long because it is a very special field (the monstrous moonshine conjecture, the fields medall of Borcherds, for exmaple was worked out in 8 years).

What would be appropriate here might be the same endorsement mechanism as for papers, to have now for comment writers. When such a section would additionally be moderated, one could set as a moderating rule to forbid

overly speculative
insulting
inapropriate comments, where inappropriate is used for comments that have not do do much with the physics involved of the paper or with physics and correct math at all.

What also could be important is a more open process when declining a paper or a comment. But therefore I have not much ideas, since the Arxiv should not discuss with cranks and open the doors for them.

At least Arxiv could
make clear rules for rejection (as statet above)
give via email a reason to the person in question why the rejection was (but only in a single sentence)
And have an address, where others (not the person in question, but maybe the endorser) can complain about the rejection and then start a new evaluation process where the person having rejected is not involved. When there is at maximum one rejection possible for one paper, the mechanism would be

1) more transparent
2) it would get rid off the cranks
3) it would allow more than paper insertion but also a commenting papers section.

Posted by: Benni on March 6, 2006 6:13 PM | Permalink | Reply to this

I think that having a specific arXiv blog is a good idea, although I think that “blogs” are not necessarily the best way to discuss things, and perhaps a “self-moderated” discussion forum as I proposed above (see my previous comment) would even be better.

W.r. to moderation, I think an “open system” is generally the best, especially if many people are involved, which would be the case since most of the scientific community actually consult the arXiv. A slashdot style system using “ratings”, or other systems based on “readers-reviewing” rather than “administrator-reviewing” (which removes a lot of load on the shoulders of the administrators and in fact generally, in my opinion, produces better results due to the large community involved in the reviewing process – take the example of wikipedia) would be fantastic.

Such ideas, I think, should be studied more carefully, and would in my opinion almost certainly lead to improvements on the current trackback system. I was very happy when the trackback idea was implemented since it gives an opportunity to discuss papers online (in such a way that people know that it’s being discussed), but taking into account the above criticism I think there is place for improvement.

In any case, I think an arXiv-based system would be much better since the scientific community (through the arXiv committee) could create it to fit exactly the needs of the community.

Best,
Vincent

Posted by: Vincent Bouchard on March 6, 2006 6:54 PM | Permalink | Reply to this

I think that there is no right to be able to place trackbacks on arXiv.org. No reasonable blogger would demand such a right. The very essence of a scientific server is that the contributions can’t be “anything”, and to a lesser extent, it is still true in the case of the trackbacks.

Jacques’ job in the committee is probably not the most pleasant one. If I were in his skin, I would probably surrender several of these cases because the trackbacks don’t seem so infinitely important to me anyway.

On the other hand, I would never, ever complain myself that my trackbacks have been deleted or something like that. Only crackpots produce similar complaints. Crackpots always believe that they are the new Galileo Galileis and whenever the established science rejects their proclamations, their feelings of being Galileo Galileis or messiahs increases even more.

Filtering is an important part of the scientific method. Some people may be very unhappy about this process, and it is more or less guaranteed that people like Peter Woit are programmed to be unhappy more often than others.

Posted by: Lubos Motl on March 6, 2006 6:31 PM | Permalink | Reply to this

Although I agree that Arxiv should delete these trackbacks and in my opinion open its own “commenting papers” section,
moderator, could you please delete the posting of motl above. The writing about crackpots, messias, and the unhappy peter he is associating with, is complete off topic.

Posted by: Benni on March 6, 2006 6:43 PM | Permalink | Reply to this

Dear Benni,

could you please kindly provide us with an explanation why you believe that my discussion about unhappy Peter and about crackpots and about messiahs who want to save science by their trackbacks is off-topic during a discussion about unhappy Peter and about the messiahs who want to save science by their trackbacks? So far your comment seems very illogical. Being the most eager eraser apparently does not yet make one a rational judge of difficult issues.

Best wishes
Lubos

Posted by: Lubos Motl on March 6, 2006 7:26 PM | Permalink | Reply to this

Motl wrote:
Only crackpots produce similar complaints.

Since peter makes such complaints you indirectly calling him a crackpot which is offensive and off topic.

Posted by: Benni on March 7, 2006 12:18 PM | Permalink | Reply to this

I’m generally disturbed at how these arxiv guidelines institutionalize one perspective on what “active research” is. Grigori Perelman published nothing for the eight years it took him to prove the Poincare conjecture; was he not an “active researcher” at this time?

There are many cultures/modes of scientific research. Some people prefer to publish many papers each year, while others prefer to work for many years until they have brought their ideas to completion before submitting for publication. Its highly disturbing that an arxiv policy would penalize researchers with high standards while rewarding those who break up their work into “minimal publishable units.”

Posted by: PM on March 6, 2006 6:40 PM | Permalink | Reply to this

The trackback idea itself seems like a good one, as it provides a way to point interested readers to informal discussion about the content and value of a paper. An alternative way to provide such a service would be to maintain a comment facility on the arXiv itself, similar in spirit to the comment section for blog posts, with posting rights being restricted to (perhaps) those who are also allowed to post papers. This latter approach also allows archiving comments in a more permanent fashion than references to external sources (blogs). But regardless of whether trackbacks or internal comments are supported, the same kinds of issues are likely to arise in deciding who can post.

That it was necessary to create a restrictive (and seemingly arbitrary) policy at the outset is puzzling to me. When setting policies, it usually seems best to watch what happens for a little while to see what problems and abuses occur, and then institute selective policies to try to curb those abuses. This seems like the preferable approach in the case at hand for the following reasons.

First, the arXiv is a repository for technical papers, and it requires a certain level of education to be able to read most of them. This in itself creates an immediate (and high) barrier that should effectively screen out most crackpots. Of course it isn’t a perfect screen, since there are also crackpots with PhD’s as well as truly vocal crackpots who will make uninformed comments based on heresay information from someone else, but given the fairly small number of people in those categories who are likely to publicly post comments, it should be quite possible to handle them on a case by case basis.

Second, most readers of a paper are likely to be educated enough to know whether commentary referenced via a trackback is insightful or just drivel or invective. ArXiv users also are likely to remember whose blogs fall into the ‘drivel’ category, and will likely not bother to follow trackbacks to them after a few disappointments. (There could even be a user rating system for a blog’s value, allowing trackbacks to be listed with those from “most valuable” listed first.)

Finally, since it seems like the entire point of the trackback system is to allow readers to elucidate or critique a paper, anyone looking at trackbacks should automatically assume that the referenced post will be less rigorous, and probably less thought out, than the original paper. Those who want more rigor can ignore trackbacks altogether or only look at those blogs they have (from their experience) identified as having an appropriate level of rigor. In a sense, this is a “let the user decide” approach.

There are ways to reduce trackback clutter that don’t depend on an arbitrary “active researcher” standard. Below is one idea. It would be more work initially to introduce than the current list of approved blogs, but it seems more adaptable and robust against arbitrariness than the current approach.

1. Have a blog registration procedure whereby a blog owner could petition the arXiv administrators for rights to post trackbacks. As part of the procedure, the owner of the blog should be clearly identified (no anonymous blog owners), and should include a curriculum vitae that would be stored on the arXiv server and accessible through a separate section of the arXiv. This would allow interested users to decide whether they think the blog owner can be belived.

2. Have uniform requirements for allowing a blog registration request to take effect. Perhaps it could be automatic for anyone would is allowed to post papers.

3. Require all registered blog owners to maintain certain standards if they are to remain registered. For example, the blog owner should be required to moderate comments to their posts, e.g., to disallow comments that aren’t accompanied by a valid email address (not to be posted), as well as deletion of all comments that are personal attacks. The goal would be to keep the signal to noise ratio acceptable.

4. Have a grievance process whereby arXiv users could post a complaint within the “blog registration” section of the arXiv mentioned in (1) above. These complaints would be viewable by any user, and would allow for viewable responses by the blog owner. No anonymous complaints would be allowed, since that would allow the same person to make multiple complaints from different machines as retribution. After some predetermined number of registered complaints, the trackback administrators could perform a quick investigation and either continue or revoke trackback priveleges. With revocation there could be automatic deletion of all past trackbacks to that blog.

The hope is to give enough freedom for the trackback idea to evolve into something truly useful. It is hard to see ahead of time what all the problems will be, so it seems wisest to allow more freedom (within bounds) at the beginning and deal with problems that actually occur, rather than trying to predict abuses at the outsent and then instituting a heavy handed approach to preclude.

Posted by: Marty |Tysanner on March 6, 2006 6:53 PM | Permalink | Reply to this

Unlike my previous comment, this one is not about my particular case, but a series of questions about the arXiv trackback policy. I’d appreciate it if Jacques or someone else on the arXiv advisory board could provide answers. For some reason, I’ve never had any luck getting answers about arXiv policy by polite e-mails to people associated with the arXiv, but have been informed about arXiv policy through blog comments.

1. I pretty regularly checked the “recent trackbacks” page over the last few months, and my impression was that by far the largest number of postings for which trackbacks to hep-th papers were generated were at the two blogs hosted at golem.ph.utexas.edu. Could some up-to-date data on this be made available so that actual data about where tracbacks are going might be used to inform the discussion here?

2. What is the arXiv policy about responding to polite e-mails from non-“active researchers” inquiring about why their trackbacks have not appeared? Will they get any response to their inquiry? Will such a response include a statement of the relevant arXiv policy?

3. How many papers must a non-“active researcher” post to the arXiv before becoming an “active researcher”? Is this number different if they’re a graduate student than if they have a significant number of publications to their name but haven’t published a paper in a while?

4. If the arXiv decides to change its current “active researcher” policy, based on discussion here or elsewhere, will this be publicly announced? Perhaps somewhere in a blog comment section?

5. Can the arXiv provide a list of “active researchers” currently allowed to have trackbacks to their blog? The “latest trackbacks” page at the arXiv today announces a trackback to superconducting.blogspot.com. This blog is anonymous and it is not clear who its proprietor is. Are trackbacks to anonymous blogs really all right? If not, can the arXiv tell us who runs this blog? E-mail to its proprietor appears to go to the owner of artilect.org, who is Douglas Bard. Is Douglas Bard an “active researcher”? Checking for his publication record by the approved method of doing an arXiv search turns up nothing.

Posted by: Peter Woit on March 6, 2006 7:37 PM | Permalink | Reply to this

### superconducting.blogspot.com

Anonymous blogs are generally not allowed. This one, if I am not mistaken, belongs to Hugo de Garis, Associate Professor of Computer Science at Utah State University.

If that’s incorrect, then it probably should not be on the approved list.

Posted by: Jacques Distler on March 6, 2006 8:42 PM | Permalink | PGP Sig | Reply to this

### Re: superconducting.blogspot.com

The blog does appear to be anonymous, but it’s plausible it has something to do with de Garis, who in January it seems co-founded “brainarchitect.org” with Douglas Bard, the proprietor of “artilect.org”, where e-mail to the blog goes. The topics of the blog are somewhat different than most of de Garis’s research. Just a guess, but perhaps it is actually a project of one of his graduate students.

Like Bard, de Garis has no arXiv publications, but he has a long publication list. His main web-page mostly advertises his recent book “The Artilect War: Cosmists vs. Terrans, A Bitter Controversy Concerning Whether Humaniyy Should Build Godlike Massively Intelligent Machines” which is published by “ETC Publications”, and which he is promoting on the UFO show “Coast-to-Coast”.

I guess I see why there is a controversy about the quality of arxiv trackbacks under your chosen moderation system.

Posted by: Peter Woit on March 6, 2006 9:44 PM | Permalink | Reply to this

### Re: superconducting.blogspot.com

A lot of computer scientists (perhaps most) do not post their papers to the arXivs.

It may well be that the (presumed?) owner of this blog is a nut. In any case, we frown upon anonymous blogs (because of the inherent problem of lack-of-accountability). So it probably should not be on our approved list, if we cannot verify ownership.

While you’re at it, any other blogs we should yank approval from?

Posted by: Jacques Distler on March 6, 2006 10:16 PM | Permalink | PGP Sig | Reply to this

### Re: superconducting.blogspot.com

“While you’re at it, any other blogs we should yank approval from?”

Well, there’s the one run by the “active researcher” Rush Limbaugh-style ranter at Harvard who seems to be one of the few people around who agrees with you that suppressing trackbacks to my blog is a good idea…

But, seriously, I’m well aware of how tricky moderation of high-quality internet sources of information is. It’s a continual problem on my blog. There I’ve chosen to err on the side of inclusiveness and let everyone who has something more or less intelligible and on-topic have their say. It’s a continual struggle to keep this under control.

I’m sure the arXiv has the same kind of struggle over postings. In general it has done a good job with this and the community as a whole is grateful. My understanding is that the wisdom of experience there is now embodied in the endorser system. Your argument that an endorser system won’t work for moderating trackback sources just doesn’t hold water. There’s absolutely no reason to believe that it would lead to a lower quality of trackbacks than your current “active researcher” system which obviously has problems of its own.

Posted by: Peter Woit on March 6, 2006 11:08 PM | Permalink | Reply to this

### Endorsements

My understanding is that the wisdom of experience there is now embodied in the endorser system.

No, that’s just half of the filtering system. The other half — the man-hour intensive, wish it were unnecessary half — is the moderation system.

We moderate paper submissions. For various practical reason, it’s unfeasible to moderate trackbacks.

The endorsement system, by itself, does not and cannot achieve the desired signal/noise ratio.

And I believe (though, obviously, cannot prove) that it would be even less effective when applied to trackbacks.

In any case, I’m puzzled that you’re so enamoured of the endorsement system (unless it’s for the purely pragmatic reason that you think it’s easy to game).

The endorsement system is rather odd, when you think about it. It resembles nothing more than an old-boy’s Club where, to become a member, you need to be “introduced” by an existing member.

For the purpose for which it was created (controlling who can submit papers to the arXivs), it is probably the optimal solution, given the constraints. But, for all the reasons I’ve stated in my post above, it’s not going to work for the trackback system.

Posted by: Jacques Distler on March 6, 2006 11:27 PM | Permalink | PGP Sig | Reply to this

### Re: Endorsements

The endorsement system undoubtedly has its faults, any system does. But commenters here and elsewhere are pointing out to you the problems with your current system. You aren’t using the dictionary definition of “active researcher”, you’re making up your own, and not able to tell us precisely what it is. It’s not getting a lot of support except from Lubos.

Everyone wants a high signal to noise ratio, but the problem here is that different people have different ideas about what is signal and what is noise. I happen to think my blog is pure signal, Lubos’s is 90% noise. He undoubtedly thinks the opposite. Each of us can find a sizable number of respectable members of the community who agree with us.

No, not by “gaming the system”, but because there is legitimately a wide range of opinion out there, much more so than at most points in history. This is because no one has a really compelling idea about how to move forward in particle theory.

What you think is “noise” other people legitimately think is “signal” (and vice-versa…). If you want arXiv trackbacks to carry the full range of interesting “signal” that is out there, you’re going to have to put up with a certain amount of noise.

Your argument that individual trackbacks can’t be moderated, so the endorsement system won’t work just doesn’t have anything to back it up. I understand you can’t moderate individual postings, so you have to evaluate blogs on their overall signal to noise level. But why should a blogger who has one or more endorsers be any more likely to produce a higher noise level than one who fits your personal definition of an “active researcher”? You should be well aware of the noise level in the literature these days. By your “active researcher” criterion, the Bogdanovs a couple years ago could set up a blog and you’d be accepting their pure noise into the trackback system.

Posted by: Peter Woit on March 7, 2006 12:25 AM | Permalink | Reply to this

### Re: Endorsements

The endorsement system undoubtedly has its faults, any system does.

I didn’t say it was faulty. I said it was half of a two-step filtering system. One half doesn’t work without the other. Why are you having trouble understanding that concept?

But commenters here and elsewhere are pointing out to you the problems with your current system.

The most prevalent opinions are that we should either

1. shut down the trackback system entirely or
2. open it to all and sundry.

Since 2 is a nonstarter, the logical conclusion would be to go with 1 …

I happen to think my blog is pure signal, Lubos’s is 90% noise. He undoubtedly thinks the opposite.

If that’s what you think, then you should suggest a trackback acceptance policy which would exclude Luboš. We certainly don’t want to be accepting trackbacks from blogs that are 90% noise, now do we?

More seriously, there’s a saying in the legal profession that “Hard cases make bad Law.” Judging the trackback system by Luboš’s 56 trackbacks (or, for that matter, Peter Woit’s would-be trackbacks) is likely to result in bad policy.

If you want to make a case for a different policy, point to some blogs (aside from your own) that are excluded under the current policy, but “should” be included. And point to some blogs (other than Luboš’s) that are included under the current policy but “should” be excluded. And then make the case that the new policy would yield a more favourable result than the existing one.

In other words, do something other than special pleading…

If you want arXiv trackbacks to carry the full range of interesting “signal” that is out there, you’re going to have to put up with a certain amount of noise.

The dominant sentiment (among those on the Advisory Board who’ve voiced an opinion) is that the noise level under the current system is already too high.

I would welcome any suggestion for modifying the trackback acceptance policy which would raise the signal/noise ratio.

Lowering it will simply get the whole system shut down.

By your “active researcher” criterion, the Bogdanovs a couple years ago could set up a blog and you’d be accepting their pure noise into the trackback system.

And do you claim that they would have any trouble finding endorsements? If so, then I know for a fact that you are wrong. If you haven’t seen any papers of theirs on the arXivs recently, it’s because of the moderation system …

Posted by: Jacques Distler on March 7, 2006 1:06 AM | Permalink | PGP Sig | Reply to this

### Re: Endorsements

You just completely ignore my point that the large problem here is that it is not so obvious what is signal and what is noise. Your current system of using an undefined term as a criterion works very well for you to optimize according to your view of what is signal and what is noise, since you get to decide who is an “active researcher”. An endorsement system would put that power in other people’s hands, some of whom have a very different view of what is signal and what is noise than you do.

I just don’t accept your terms of setting the problem that the question is how to further reduce the noise level. The problem instead is that there is too little real useful signal. I’m not worrying about how to suppress the noise coming from Lubos, he’s only one of a large number of noise sources in the particle theory community at the moment, and at least at times he’s kind of amusing.

If you’ll give me a list of bloggers who discuss hep-th papers who are not “active researchers” and are having their trackbacks rejected, I’ll be happy to tell you which ones I think have a useful signal and could find endorsers. I don’t know at all who is on this list, maybe it’s short, maybe it’s long, maybe it’s all crackpots. I was just over at the blog “Life on the Lattice” of Georg von Hippel, if he’s not an “active researcher”, I guess he’d be an example.

I obviously don’t know enough about the endorsement system to know why you have people endorsing the Bogdanovs. If you do, your moderator is acting as an override of a broken endorsement mechanism, trumping a bad endorser. You could still have the same override mechanism in the trackback case. The problem with the Bogdanovs isn’t that some of their papers are good and some are bad, so they need to be individually moderated.

Posted by: Peter Woit on March 7, 2006 1:45 AM | Permalink | Reply to this

### Re: Endorsements

I just don’t accept your terms of setting the problem that the question is how to further reduce the noise level. The problem instead is that there is too little real useful signal.

Again, name a blog that you read (aside from your own) which would contribute to the “signal”, were its trackbacks not blocked.

I was just over at the blog “Life on the Lattice” of Georg von Hippel, if he’s not an “active researcher”, I guess he’d be an example.

Life on the Lattice is on the approved list (by virtue of its previous owner, Matthew Nobes). Georg may just be starting his research career, but he has produced two papers this past year (and a conference proceedings) so it seems reasonable to keep him on the list.

I obviously don’t know enough about the endorsement system…

That about sums it up, dunnit?

Posted by: Jacques Distler on March 7, 2006 2:14 AM | Permalink | PGP Sig | Reply to this

### Re: Endorsements

Good to hear we are on the approved list.

My two or three attempts to post trackbacks to the arXiv have failed, though, so I suppose there is some technical trouble either with the trackback system (when I click on the “recent trackbacks” link on the arXiv homepage, I get a “cyclical link” error in my browser [konqueror], btw) or at my end (I use blogger, so I had to try and post the trackbacks manually).

Posted by: Georg on March 7, 2006 8:02 AM | Permalink | Reply to this

### Re: Endorsements

“That about sums it up, dunnit?”

Always a pleasure to try and have a serious discussion with you, Jacques.

My mistake last night, because of the lateness of the hour, of getting into a side discussion here, one that has nothing to do with what is really at issue.

What criterion you use to moderate trackbacks is besides the point here. My claim is that, under any moderation system designed to identify blogs with a high signal to noise ratio and block those with a low one, my blog should pass that test. If you want to use an “active researcher” criterion, fine. I am one, as I have argued in detail here. The problem with your “active researcher” criterion is that you have left it ill-defined, and are abusing it because you don’t happen to like my scientific views. I’m not the only one pointing out this problem to you.

There are actually two quite different sorts of “noise” that are at issue here. One is bad, uninformed science, but the other is incivility. I see that Joe Polchinski has written in, and the civility issue is part of the problem he is identifying. Whatever the arXiv policy is, “active researcher”, endorsement, or something else, some mechanism should be found to strongly discourage the kind of incivility that is too prevalent in these blogs. Lubos is by far the worst offender, but, with your continual attacks on my competence, Jacques, you’re not far behind. I’ve undoubtedly made mistakes in the past in responding to this kind of incivility. That’s kind of the problem with allowing anyone to start it, it tends to spread.

Posted by: Peter Woit on March 7, 2006 8:42 AM | Permalink | Reply to this

### That’s it …

What criterion you use to moderate trackbacks is besides the point here.

No, it is the only point here. The particulars of your case are beside the point.

I see that Joe Polchinski has written in, and the civility issue is part of the problem he is identifying.

No, he was complaining that “This post clearly has no science content.” and hence that “the arXiv, should [not] be providing pointers to [it].”

Whatever the arXiv policy is, “active researcher”, endorsement, or something else, some mechanism should be found to strongly discourage the kind of incivility that is too prevalent in these blogs.

I agree. If someone could suggest a mechanism for doing that, I’d be all ears.

…but, with your continual attacks on my competence, Jacques, you’re not far behind.

In the 3 1/2 years that I have run this blog, I have mentioned your name a grand total of 4 times. (Which, apparently, is not frequently enough for you.)

So much for my “continual attacks.”

You, by contrast, bring up my name with alarming regularity, (96 times, as of this moment) invariably coupled with some insult or accusation.

I don’t know why you’ve taken such an intense dislike to me. While I can’t do anything about what you say about me elsewhere, I’m not in the mood to tolerate your wild accusations over here.

Since you have repeatedly violated the ground rules that I set out for comments on this post (stick to a discussion of the general policy, not the specifics of your own case), I am, very regretfully, going to ask you to take the rest of your comments elsewhere.

Posted by: Jacques Distler on March 7, 2006 9:39 AM | Permalink | PGP Sig | Reply to this

### Re: That’s it …

Before this trackbacks issue arxive was a nice place.
No childish offensive comments, no jokes about papers, no questions what is a scientifically appropriate comment, no questions what an active researcher is
and no questions if a moderator has scientific and reasonable rules for blocking a trackback or if he would personally judge.

This was before the trackback system came. That means:
To shut it down is the best.

BTW:
Why does Arxiv board member ethan write at cosmic variance, that he
“does not want to hazard a guess, wether the blog of woit has been the reason for this “active researcher” criteria”?

Why does he write at cosmic variance
“If you think this is not an appropriate criterion, send a note to the advisory board. Public feedback is good”?

Why does he write
“Peter, I don’t consider you as a crank. If you think you are an active researcher, I would strongly oppose against this decision to the arxive board”?

Why does Ethan do this?
From an outside position, and as an astronomer, ethan has such one, this discussion looks quite strange.

If one would find a mechanism, which looks only at the question: “are these blog entries purely related to science and an appropriate discussion of the literature”, then many entries of peters blog might be allowed.

But there is no such mechanism. So everytime a moderator rejecting a trackback would be in danger of being called a censor from those who like to read the blog in question.

What, if everyone with a phd. and an university affiliation could send trackbacks from blog?

What if we say, only entries that comment papers and are commited to science only are allowed?
Then we would have many authors which are insulted from short hypocritical comments, and there would be flamewars on critics about papers starting out which are not appropriate or wrong.

There is only one solution:
No trackbacks at all in a scientific archive.
It is to much personal information there in Blogs. This can’t be moderated.

Posted by: Benni on March 7, 2006 12:49 PM | Permalink | Reply to this
Weblog: Uncertain Principles
Excerpt: The comment I would post at Jacques Distler's blog, if I were allowed to post comments at Jacques Distler's blog.
Tracked: March 6, 2006 8:11 PM

I would like to write in support of an arXiv policy that restricts trackbacks, but a different policy than the one that is actually in place. In support of my argument I quote in full one blog post regarding my Scientific American article with Raphael Bousso, The String Landscape:’

“One article in the magazine doesn’t really have much to do with Einstein and I believe would make him gag if he were still around. The article, entitled “The String Theory Landscape” is by Raphael Bousso and Joe Polchinski. In it they claim credit for the pseudo-scientific idea of “explaining” the value of the cosmological constant by the existence of the “landscape” and the anthropic principle. It’s sad to see this nonsense being purveyed by the most respected and well-known popular science publication in the US.”

This post clearly has no science content. To those who would quickly retort the landscape has no science content,’ I offer as counterexample the paper Is There a String Theory Landscape?’ hep-th/0309170 by Banks, Dine, and Gorbatov, which is a set of well reasoned scientific challenges to the landscape idea.

The arXiv is a repository for serious scientific work and discussion. Statements such as this would make Einstein gag,’ and so on are not a part of a serious scientific discussion and I do not believe that the science literature, i.e. the arXiv, should be providing pointers to them.

However, I am not sure that there is a defensible policy that singles out individual blogs; rather, I am skeptical of the trackback idea. It is true that the `permitted’ blogs do in general discuss the scientific content of the papers that they point to, though not always in the most reasoned fashion. The problem is that blogs are ephemeral. Nobody blogs about last week’s paper; the emphasis is on doing it fast not doing it right. A fraction of posts have some lasting value, but more in the nature of a trivia collection than a scientific literature. I do not see the sense of cluttering the arXiv with a set of permanent pointers to these quick-hit remarks. Writing a paper is a serious undertaking, and any response recognized by the arXiv should be equally serious.

Blogs are fine, and I waste time reading them than I would like to admit, but if trackbacks are to be implemented I believe that this should be in the form of some independently maintained overlay to the arXiv rather than the arXiv itself.

Posted by: Joe Polchinski on March 7, 2006 1:06 AM | Permalink | Reply to this

### Old News

Nobody blogs about last week’s paper

Actually, I do, but that’s because I am perpetually behind …

More seriously, I think there’s a fair amount of value clustered near the top of the table that I posted above.

But I seriously wonder whether it is worth a fraction of the time I have wasted on this faux controversy.

Posted by: Jacques Distler on March 7, 2006 1:24 AM | Permalink | PGP Sig | Reply to this

Joe,

I’m the author of the posting you quote and would like to make several comments about it. First of all, you’re right that the tone is quite objectionable and I’d like to apologize to you for it. I sincerely regret writing that posting in that way, and you’re right to object to it.

But while I agree with you about the civility issue here, I disagree with you about the issue of whether this kind of thing is legitimate scientific discussion. The example you give is a low point for my blog, but it also isn’t directly relevant to the debate here. That posting is not about a scientific article posted on the arXiv, contains no links to arXiv material and would never generate a trackback. One reason it doesn’t have a detailed scientific argument is that it was responding to a popular science article designed to promote something to the public, not a research article.

By now my blog contains a large number of postings that do directly address research articles on the landscape issue. I’m sure some of these postings have similar civility issues to those in the one you quote, but I believe that on the whole they contain serious scientific arguments about the research papers involved, often going into some detail. If you want to discuss this question further, I could go back, pick out some of my postings on this topic that do link to arXiv papers, and explain the argument being made there. The issue of whether the string theory anthropic landscape is science or not is a very real one, and a very important one for the physics community to address. One reason my and other blogs have been getting a lot of attention is because they are a place where this issue is getting addressed, even if it is too often in an incivil way.

I actually think that blogs are a highly appropriate place for this kind of discussion. When a “landscape” paper appears containing arguments that make no scientific predictions, and that inherently can’t be used to make scientific predictions, what is the best way to raise this issue? Should I or someone else write a paper for the arXiv that goes through the original paper, showing in detail why its arguments can’t possibly ever lead to real predictions, or is this something that is best done on a blog, then linked via the trackback mechanism to the arXiv posting? I happen to think that the first alternative really would clutter up the arXiv with a lot of noise, the second is a much better way of dealing with this.

Again, my personal apologies for the tone of the blog posting you quoted, and for any other similar ones I may have written. The one you quoted was written early on in the history of this blog, during the summer of 2004. I hope that I’ve learned some things since then and have generally stuck to a more civil tone, and will endeavor to be even more careful about this in the future.

But your comment makes clear what the real issue is here. It’s not about whether I’m an “active researcher” or not, it’s about the legitimacy of forceful criticism of a currently very popular research program in particle theory.

Posted by: Peter Woit on March 7, 2006 10:34 AM | Permalink | Reply to this

Dear Peter,

I appreciate the apology, but I have to question its sincerity when you use it as a jumping off point for a number of non sequiturs that largely ignore the content of my post. In particular, I fail to seem much connection between your final paragraph and what I have said. In fact I do prominently cite a scientific criticism of the landscape. I invite you or anyone to read hep-th/0309170 alongside whatever you regard as your most scientific statement on the landscape. I think that it will be evident that they are very different things, each of which may have its place, but the arXiv is the place only for the former.

Posted by: Joe Polchinski on March 7, 2006 12:31 PM | Permalink | Reply to this

I remember, that not long ago, I wrote a comment to peter’s blog, which I wrote exactly the same argument (with other papers). It might have peter made a bit angry, so he deleted my comment…

Posted by: Benni on March 7, 2006 1:25 PM | Permalink | Reply to this

What Einstein would do, is indeed not gag. This is a typical impoliteness, which frequently on those blogs occur and that’s why these trackbacks should be shut down.

From Einstein, we know that he sent very appreciating comments to Kaluza because of his “courageous” idea. But he did so because he tried to forward young researchers generally.

Personally, Einstein himself wrote in his lecture notes at princeton:

“I think that speculative ideas like extra dimensions should be only taken into reaerch when there exists a reliable experimental evidence for it”.

Clearly, Einstein said: Without experimental results it is better to stop any theoretical investigation relying on purely mathematical terms. That is he would be strongly opposed to any quantum gravity program and look at the foundations of quantum mechanics and quantum field theory until they are settled down.

Posted by: Benni on March 7, 2006 1:16 PM | Permalink | Reply to this

Dear Jacques Distler,

My suggestion would be:

or

2) allow trackbacks from approved registered researchers.

To be approved, the researcher must:

- have a PhD in Physics, Mathematics, Astrophysics/Astronomy or related area
- provide academic or research institution affiliation
- provide a brief CV

PhD students could also register by sending:

- a simplified recommendation letter from his/her advisor (perhaps just an email communication from the advisor to the arXiv board could suffice)
- provide at least 2 publications in refereed journals

(The above information could be entered by the user in a simplified on-line form provided by the arXiv).

Allow a regulatory system, so that the registration should be renewed every year by the participants. Registered researchers would also be allowed to send a communication to the arXiv board by the end of the year (with deadlines clearly indicated) informing abuse or objectively arguing why a given blog/researcher should not have his registration renewed. In this case, the standards and rules of the arXiv must be clearly defined so that anyone is capable of understanding what an abuse is.

The current registered researchers/students credentials should be publicly accessible from a separate arXiv page.

Now, if you allow me, I have a few questions.

1) From the current system, I am an endorser to astro-ph.
Can I send trackbacks from my blog to quantum gravity papers (which, as you know, are mostly submitted to the gr-qc/hep-th category)? I have only one paper in gr-qc. I am just starting research in this area. I truthfully believe my blog provides informative and constructive discussions, at least I work to that aim. I do not allow personal attacks in my blog.

2) My blog is hosted at blogger. According to the
help pages, Blogger does not currently support Trackback. Is there another way I could send trackbacks to the arXiv from my blog?

Thank you very much,

Christine

Posted by: Christine Dantas on March 7, 2006 5:57 AM | Permalink | Reply to this

2) My blog is hosted at blogger. According to the help pages, Blogger does not currently support Trackback. Is there another way I could send trackbacks to the arXiv from my blog?

You can get an account at Haloscan which provides trackback features on top of blogger.com. At least, this is what I did for atdotde when trackbacks were introduced for the arXive. Still, the actual procedure is quite a pain in the neck.

Posted by: Robert on March 7, 2006 6:26 AM | Permalink | Reply to this

Thanks, Robert, I will try that.

Best wishes
Christine

Posted by: Christine Dantas on March 7, 2006 6:42 PM | Permalink | Reply to this

The idea that the approval of a given blog to send trackbacks ought to be reviewed at some regular interval is a good one. Right now, the system is too new for us to have run into that issue yet.

Currently, we’re still bickering about the criteria to be approved the first time.

From the current system, I am an endorser to astro-ph. Can I send trackbacks…

Based on your publications in astro-ph, you’d qualify. (I realize you are in the process of switching fields, which raises various other issues we haven’t encountered yet … This stuff only looks easy from the outside!)

Posted by: Jacques Distler on March 7, 2006 7:26 AM | Permalink | PGP Sig | Reply to this

Dear Jacques Distler,

I realize you are in the process of switching fields.

Not at all. I do have regular collaborative work in Astrophysics/Cosmology (BTW, I hope to post in a few months a new paper in astro-ph. I only post there papers after accepted for publication in refereed journals). I am currently dedicating about 20% of my time in Quantum Gravity. It is too early to know what will be the outcome of my studies, but I do hope I will be able to include QG into my regular research lines. It depends on several variables, but in any case I am taking this seriously as can be seen in my blog.

Thank you very much,
Christine

Posted by: Christine Dantas on March 7, 2006 8:11 AM | Permalink | Reply to this

Maybe this is too complicated to implement, but how about it: Each arXiv user gets to put the URL to their blog or homepage into their arXiv user profile along with their email address. Each time someone posts a paper, they receive a number of trackback credits (five, say), which can then be used to post trackbacks to papers. No credit, no trackbacks.

This would formalise the “active researcher” criterion in an objective manner, while being inclusive of researchers with short publication records and keeping the signal-to-noise ratio high, since you wouldn’t want to waste your hard-earned credits.

Posted by: Georg on March 7, 2006 7:53 AM | Permalink | Reply to this

Also, I agree with the person who said that blog posts are somewhat ephemeral, while the arXiv is permanent. So maybe backtrack links should be removed after a fixed time (a year, say) when they will no longer be of interest.

Posted by: Georg on March 7, 2006 7:55 AM | Permalink | Reply to this

### Into the Future

So maybe backtrack links should be removed after a fixed time (a year, say) when they will no longer be of interest.

There are lots of occasions where an old blog post (if well-done) can serve as a useful adjunct to a paper long into the future.

• Sometimes a paper is wrong, for some subtle (or not-so-subtle) reason. You might think, “Oh, someone will write a followup paper, pointing out the problem.” But, often, that doesn’t happen. The literature just moves on, and the paper is left behind as a little landmine for someone to stumble on later. Perhaps no one thinks it worthwhile to write a paper; but maybe a blog post will do. If so, that blog post remains useful far into the future.
• Maybe it’s not an error, but some other subtle point in the paper which a blog post might help elucidate. Or maybe it provides a comparison between the approaches of different papers to a certain problem.

All of these things (you can make up more examples for yourself) are things that may be well-understood by the experts on a given topic at the time of writing. But for someone coming to the subject afresh, perhaps years later, it would be very helpful to have that “common knowledge” jotted down someplace.

That’s the sort of role that blogs (and, if the commenters are good, the associated comment sections) can play. And, at its best, it actually becomes more useful with the passage of time.

Posted by: Jacques Distler on March 7, 2006 8:20 AM | Permalink | PGP Sig | Reply to this

### Re: Into the Future

It’s true that some old blog posts have the potential to be valuable resources. But one has to be worried about blogs becoming defunct or moving (e.g. because their authors leave the field or move to another university), leaving the arXiv with dead links.

Posted by: Georg on March 7, 2006 9:54 AM | Permalink | Reply to this

If the papers posted to arxiv were only on physics topics and methods, then the policy might be ok; however, physicists frequently post scientometric or bibliometric articles as well as articles on social network analysis. For example, “Journal Status” by Johan Bollen, Marko A. Rodriguez, and Herbert Van de Sompel, arXiv:cs.GL/0601030 (sorry for poor citation format), is an article properly in the field of bibliometrics. The experts in this field (Leydesdorff, Bensman, Goodman, Garfield, et al) do *not* publish on arXiv. They publish in JASIST and Technometrics and other places. The are the most qualified to review this article and they are having a fascinating conversation on an ASIST mailing list. Shouldn’t this conversation be linked to the original article? I would argue that their conversation is more appropriate than some pointers from a physicist who hasn’t studied citation analysis. I think it should be more democratic to be useful. Otherwise, how about something like http://postgenomic.com/ for physics?
BTW- ASIST should properly be ASIS ampersand T but that’s not “valid”

Posted by: Christina Pikas on March 7, 2006 8:24 AM | Permalink | Reply to this

In that sense, i think that Physics Comments does a similar job, doesn’t it?

Posted by: Daniel Doro Ferrante on March 7, 2006 8:46 AM | Permalink | Reply to this

Physics Comments was designed to do a similar job, and in fact when well identified authors comment on an arxiv paper, the arxiv trackbaks them (if the author does not identify properly in the first line, we do not ping arxiv, even if s/he is a registered user).

Now, the truth is, nobody cared about Physics Comments. Physicists getting involved in the blog thing prefer to set up their own blog (or to ask staff to do set up it). So only a handful of users have bothered about registering and uploading comments. So it is clear that the trackback mechanism must be in the arxiv itself, in order to accuratelly collect all these scattered blogs around.

(yeah, we have more than a handful of comments, but this is because during the first year authors were invited to blog anonymously for any (or some) preprint they were referenced in. This mechanism was discontinued time ago)

Posted by: Alejandro Rivero on March 8, 2006 7:15 AM | Permalink | Reply to this

If the papers posted to arxiv were only on physics topics and methods, then the policy might be ok; however, physicists frequently post scientometric or bibliometric articles as well as articles on social network analysis.

Who said only papers on the arXivs count? The best measure, still, is papers published in refereed journals.

In fields like high energy theory, everyone submits their work to the arXivs. That is not true in other fields, and one would imagine that the arXiv adminstration is smart enough to have taken that into account.

Leading researchers in bibliometrics would certainly be permitted to post trackbacks to papers on the arXiv on that subject.

…they are having a fascinating conversation on an ASIST mailing list. Shouldn’t this conversation be linked to the original article?

Mailing lists and web forums are very dicey in terms of generating acceptable trackback links, because it’s hard to determine who can contribute (and, in terms of mailing lists, the mailing list software is simply not set up to generate trackback pings).

One example of a web forum that is allowed to post trackbacks is cosmocoffee, because they have rather rigorous standards for membership. Most web forums and (I imagine) most mailing lists would not be permitted.

BTW- ASIST should properly be ASIS ampersand T but that’s not “valid”

“&” is spelled “&amp;”.

Posted by: Jacques Distler on March 7, 2006 8:52 AM | Permalink | PGP Sig | Reply to this

In fields like high energy theory, everyone submits their work to the arXivs. That is not true in other fields, and one would imagine that the arXiv adminstration is smart enough to have taken that into account.

But that raises the question of why anyone who doesn’t bother to publish to arXiv would ever bother to go through a convoluted process to submit trackbacks there. Yet articles outside the field of expertise of the majority of readers are exactly those where more informal discussion of the topic in blogs would be most helpful for those reading the paper.

Posted by: jgraham on March 7, 2006 11:33 AM | Permalink | Reply to this

### An email from Hirosi Ooguri

Hirosi sent me the following comment via email.

Dear Jacques,

I enjoy reading your scientific commentaries on papers posted on the arxiv. I think your web site is an excellent venue for that.

When I prepare a paper for the arvix, I tried to write it so that it best represents what I have done. I may not be alway successful, but I do not wish any casual commentary attached to it directly, which may misrepresent what I have done, even if it is written by an active researcher.

I think that it is good to have variety of venues with different levels of scientific rigor. I would like to see the arvix keep its high
standard by limiting contributions to papers reporting original scientific discoveries and scholarly reviews.

It would be useful to have a place for less formal scientific exchanges. I think it should be at a venue that is separate from the arvix.

Sincerely, Hirosi Ooguri

That authors would object to their abstracts being linked to commentaries (possibly unfavourable) about their paper is one of the reasons no commentary layer has ever been created at the arXivs.

It is also the reason that the trackback links do not appear on the abstract page itself, but on a separate page (containing strong disclaimers). I realize it’s only a figleaf, but it does mean that arXiv users have to actively seek out the trackback links, rather than have them staring up at them from the abstract page.

With the help of Google, it’s not terribly hard to find websites which comment on arXiv papers, even if there is no trackback link at arXiv.org. Unfortunately, Google makes absolutely no guarantees of quality-control.

The arXiv trackback policy, if implemented well, provides at least some measure of filtering. Given that people are going to publish online commentaries, and given that potential readers are going to seek out those commentaries, it is to the author’s advantage to steer them in the direction of the most informed and responsible of those commentaries.

I think the overall quality of such commentaries (my own, included) could be higher. Time will tell whether that quality does improve. If it it doesn’t, then I think we can all conclude that the experiment has been a failure.

Posted by: Jacques Distler on March 7, 2006 10:03 AM | Permalink | PGP Sig | Reply to this

Well, It seems that the current policy doesn’t work the way it was intended.. To say the least :^)

One might also exercise moderation via a classification scheme.

A- Detailed Research Level Discussion blogs.
C- General Public Popular Science blogs

If a blogger wants a higher recognition then he can simply improve his postings. (More constructive than flame wars)

Which blogs really deserve A status anyway?

Also, which author would not want to know if his or hers personal work is mentioned on a popular science site like Scientific American or New Scientist? Even if it’s only to be able to go over there to clarify his/hers work.

Regards, Hans

Posted by: Hans de Vries on March 7, 2006 11:45 AM | Permalink | Reply to this

Hans’s post seems like the best option to me. Take what ever objective criteria you want. Personally I’d choose something along the lines of has a research position at a credible university, be it a grad student, post-doc, scientist or professor. Then simply ask the owner of the blog to define the purpose or content level of the blog. I’m guessing few, if any, will actually lie. Pick the level of discourse, say A or B above, and only allow track backs from those sites. A modicum of moderation on a blog-by-blog basis (and not a post-by-post basis) would likely be required to ensure the blog owners don’t lie, but this is probably no more than ensuring the blog owners are active researchers or anything else.

As an example, from the point of view of the arxiv, I don’t see why the writers of Cosmic Variance (as much as I enjoy that particular blog) need to have their trackbacks registered. Their posts, even when they involve papers on the arxiv, are intended to educate and appeal to a general, albeit physics minded, audience, not an audience of active theoretical physicists. Such a policy would reasonably exclude Cosmic Variance as well as possibly Peter’s and Lubos’s blog, neither of which strike me as being worthwhile for research purposes from the arxiv point of view, even if they are worthwhile from my point of view.

Otherwise, yeah, I’d drop the whole thing altogether.

Posted by: Anonymous on March 8, 2006 8:59 PM | Permalink | Reply to this

My understanding is that WordPress, the software used by Cosmic Variance, does not automatically send trackbacks. So if one of them decided that a particular post merited sending a trackback to the arXivs, do we really want to say, “No, Sean. That post of yours isn’t really worth sending a trackback.”

Posted by: Jacques Distler on March 8, 2006 10:43 PM | Permalink | PGP Sig | Reply to this
Weblog: NeverEndingBooks
Excerpt: If you happen to have a couple of hours to kill, you might have a look at the arXiv trackback policy debate over at Jacques Distler’s blog Musings. But before you dive into this it is perhaps useful to glance at what went before. Distler did pes...
Tracked: March 7, 2006 2:02 PM

Since I am currently on the approved list according to Arxiv policy for astro-ph, im going to chime in my two cents.

I very much like the idea of trackbacks and would like to see them continue. I respectfully disagree with colleagues who do not like such errata for their papers.

I say this b/c I recognize a growing and irratating problem with arxiv in general, namely the proliferation of papers that are either erroneous, grossly speculative or more or less rehashes of previous work (often from decades earlier).

For instance the Sarfatti paper on hep-th that was linked recently, had a reply to it that more or less thoroughly debunked the paper. This could have been accomplished far more efficiently with a blog posting (b/c in essence pointing out a damning error can be done in such a forum, whereas actually providing new fleshed out research can only be done with a paper). Now, for less ridiculous papers, this is obviously harder, but I think still fair game and desirable in the long run.

Indeed this goes beyond arxiv, there are plenty of sensible blog and newsgroup posts that debunk accepted peer reviewed papers that somehow passed the referree stage. It would be nice if we have historical comments for future researchers to see whether

a) This paper is outdated
b) It has been challenged
c) It was wrong
etc

Now as far as current policy per se, let me just say I am conflicted by it. I would prefer if the ‘active researcher’ standard worked the other way around, putting less burden on potential blog criticism (and having to deal with noise and potentially bad critiques) and more on the actual paper to begin with. I mean doesn’t that seem more logical, the burden of proof should be harsher on the submitee?

Anyway, my gold standard for scientific blog posting is the string coffee table. If everything was done with that attitude and carefulness (for instance the emphasis on mathematics rather than english) as well as the frequent posting and high signal to noise ratio, I very much doubt we would be even talking about this problem

Posted by: Haelfix on March 7, 2006 3:35 PM | Permalink | Reply to this

### Sarfatti

For instance the Sarfatti paper on hep-th that was linked recently…

Please! That was gr-qc. The moderators for hep-th are evil censors of such heterodox ideas :-)!

If everything was done with that attitude and carefulness … I very much doubt we would be even talking about this problem

I’m afraid we will always have such problems. Some fraction of the people whose papers are refused by the arXivs complain (sometimes very loudly). There’s no reason to expect trackbacks to be any different.

The ability of bloggers to make a lot of noise, however, is a qualitative change. If the trackback system on the arXivs persists, I can guarantee that this will not be the last such controversy.

Posted by: Jacques Distler on March 7, 2006 3:58 PM | Permalink | PGP Sig | Reply to this

### Re: Sarfatti

compared to science, blogs are generally noise. As Polchynski said.

What if I post a paper, something does it not understand, and writes wrong critics on a blog which is censored by him (that is I cannot reply without having an own blog) and gets many (2000) hints a day.
This feature could easily destroy carriers.

We should shut down this trackback feature immediately.

Posted by: Benni on March 7, 2006 5:19 PM | Permalink | Reply to this

As might have been stated before, the problem is not the existence of a particular trackback policy. It is that

1) it was not publicly promulgated, and
2) it is not objective.

Had an objective policy been clearly posted on arXiv, the controversy would not have arisen.

I suggest that it IS in fact constructive to explicitly define an “active researcher”, being fully aware that any definition will be imperfect.

Few will object to defining a minimally active researcher as one who has posted an average of 2 papers per year to arXiv over the last 3 years.

Adopt such an explict standard, publicize it, understand that you can’t please everyone, and move on.

Posted by: Belizean on March 7, 2006 3:41 PM | Permalink | Reply to this

Belizean,

One effect of the policy you suggest is that it would exclude Jacques’s blog, since he has only posted 3 papers during the past three years. I suspect he’s not going to go for your “active researcher” definition.

Posted by: Peter Woit on March 7, 2006 4:50 PM | Permalink | Reply to this

“Which blogs really deserve A status anyway?”

Well, Andy Neitzke’s blog at UCSB comes to mind:

https://blog.kitp.ucsb.edu/weblog/strings05/

Detailed and mostly accurate description of scientific content. Period.

I enjoyed it a lot. Is this blog on the “approved” list? Unfortunately sometimes the best candidates do not even apply…

Posted by: Michael on March 7, 2006 9:27 PM | Permalink | Reply to this

I think there is some sort of an antinomy in the “active researcher” definition: Active researchers tend not to write blogs. There are certainly exceptions to the rule but “active researcher” and blogger is negatively correlated in my opinion.
As a user of arXiv my recommendation would be to shut down the trackback system and use the resources of arXiv to improve the moderation system instead.

Posted by: wolfgang on March 8, 2006 7:03 AM | Permalink | Reply to this

Dear Jacques,

Not so long ago I was asking the arXiv Adisory Board about the possibility of an arXiv based commenting feature instead of the trackback system. From your current post I understood you clearly saying “not gonna happen” and followed the link you provided because I “wanted to understand why”. However, I only found this sentence:

“For understandable reasons, Paul Ginsparg has been reluctant to host any such endeavour at arxiv.org itself, wishing to maintain it purely as an e-print server.”

I also did not receive an answer to my email I sent to the Advisory Board (well, I guess they are busy).

For this reason I would like to ask you here about the understandable reasons against an arXiv based commenting system, or please point me to the appropriate material where I can read and understand the rationale behind Ginsparg being very reluctant about it.

Best wishes,

As you can see, some authors object even to a trackback link to an off-site website. An arXiv-sponsored discussion about their paper would be beyond the pale.

Others might find trackbacks acceptable, but feel strongly that the arXivs themselves should be strictly a repository for scientific papers.

Moreover, policing such a discussion forum would be an ongoing nightmare, as you can see from the wrangles associated to the much more innocuous trackback system.

For all those obvious reasons (and more besides), any such arXiv-sponsored discussion forum is out of the question.

Posted by: Jacques Distler on March 8, 2006 1:56 PM | Permalink | PGP Sig | Reply to this

Dear Jacques,

Thanks for the reply. Having understood your point my vote - if voting on this issue counts and this is the write place to announce it - is to shut the trackback feature off.

The argument for a strictly scientific repository is a winning one against comments and is also a winning argument against trackbacks.

Best wishes,

Jacques: just a question about the moderation of the arxiv. I couldn’t find any info on the main site, but are all archives moderated or is it only hep-th? (Your comment about gr-qc could be interpreted in that way.) I’m only submitting to hep-ph and have never seen any signs of moderation (eeh, that didn’t sound right…).

Posted by: Rien on March 8, 2006 1:58 PM | Permalink | Reply to this

The physics moderators are listed on this page which, for other reasons, it would be in the interest of all physics arXiv users to read.

These are the quant-bio moderators. Somewhere or other are pages with the CS and math moderators. I’m too lazy to track them down.

Posted by: Jacques Distler on March 8, 2006 2:19 PM | Permalink | PGP Sig | Reply to this

Thanks, I must’ve not looked very carefully. It strikes me that this must not be a very desirable job and surprises me that there aren’t more of them.

Posted by: Rien on March 8, 2006 3:57 PM | Permalink | Reply to this

### The life of a moderator

Given the grief I’ve gotten from some quarters, I’d have to say that the job does have its downsides.

Posted by: Jacques Distler on March 8, 2006 4:07 PM | Permalink | PGP Sig | Reply to this

Maybe it could help indentifying interesting papers, but, here is an example of a blog entry from a person stated as an active researcher:
http://motls.blogspot.com/2006/03/theory-of-everything-from-trinions.html
If I would be smolin, I would recognize at least the picture at the end of this Blog entry as an offense

That´s an important point: why trackbacks REALLY offensive like this one are allowed, while others are not??

Posted by: Anonymous on March 8, 2006 5:10 PM | Permalink | Reply to this

Because, if you’d bothered to read my post, I explained that

1. We are not (and cannot) moderate individual trackbacks. Acceptance of trackbacks is on a blog-wide basis.

2. I don’t know how, in any impartial and practical way, to enforce standards of decorum on those blog posts.

If you can think of a mechanism for doing that (which would not involve Cornell staff reading all the blogs which post trackbacks), then please take the time to make a proposal.

I’d love a mechanism for eliminating trackbacks from posts which contain intemperate language, sloppy scientific thinking, etc. But I don’t think such a mechanism exists.

Our filter deals only with the qualifications of the author, not with the content of individual posts.

But I seem to be repeating myself. Do I have to add Luboš to this list of topics that are off-limits in this comment thread?

Posted by: Jacques Distler on March 8, 2006 5:26 PM | Permalink | PGP Sig | Reply to this

Why not simply turn trackback moderation over to authors (except for spam deletion and so on)? Of course some authors wouldn’t want to deal with this, so there could be a box that one would have the option of checking upon submitting the paper that would just prohibit all trackbacks (or allow them all). But I suspect that most authors would want to read blog postings about their papers anyway, in light of which it wouldn’t be much more trouble for them to approve or deny trackbacks from each individually.

Posted by: mike on March 9, 2006 1:02 PM | Permalink | Reply to this

I’d like to pursue mike’s suggestion. Would it be possible to allow the author of an article make the decision as to which trackbacks to allow?

Posted by: Daryl McCullough on March 9, 2006 1:50 PM | Permalink | Reply to this

### Let the author decide

That’s certainly one thing we are looking at.

It doesn’t solve all problems. You would still need some filtering. If any crackpot with a website could post trackbacks to the arXivs, that would quickly pose an undue burden on the authors. Most would (I certainly would) choose to turn trackbacks off, rather than have to deal with the continuous stream of graffiti.

It also would tend to eliminate any trackbacks which point out, say, that a paper is wrong. One presumes, from the reader’s point of view, those would be the most useful kind…

Posted by: Jacques Distler on March 9, 2006 2:14 PM | Permalink | PGP Sig | Reply to this

nothing to say, but what is the reason for an approval to comments here?
I don’t see any unwanted material here

Posted by: blubblab on March 8, 2006 7:33 PM | Permalink | Reply to this
Weblog: 格志
Excerpt: 前几天物理 blog 圈里有对 ArXiv 的 Trackback 政策的热烈讨论。起因是质疑弦论的 Not Even Wrong 的 Peter Woit 对此的疑问，经过 Sean的评论，引起很大的反响。Jacques Distler 于是出来解释了一下 ArXiv 的 ...
Tracked: March 9, 2006 9:32 AM

### A proposal

Dear Jacques,

You wrote in response to someone else’s post:

“If you can think of a mechanism for doing that (which would not involve Cornell staff reading all the blogs which post trackbacks), then please take the time to make a proposal.

I’d love a mechanism for eliminating trackbacks from posts which contain intemperate language, sloppy scientific thinking, etc. But I don’t think such a mechanism exists.”

Your second sentence confirms to me that the trackback system should be shut off. If it can not be operated in a reasonabe way – which is in essence what you are stating – then why operate it at all? In addition, there is at least one blog which features text ads from Google, thus the traffic to the site generated by the trackbacks on arxiv.org directly contributes to the financial well-being of the blogger. What is your and/or the Advisory Board’s view on the fact that it is possible for some arxiv.org users to generate income as a direct consequence of the trackback system?

Since based on your first sentence it seems you welcome proposals, please take a moment – and anyone else reading – to comment on mine (if something similar has already been proposed and I only missed it then please ignore my post and point me to the appropriate place):

It seems there is consensus among all parties involved that SOME of the functionality that the trackback system provides is useful (I call this the first consensus). There is also consensus on the view that the arxiv.org site should remain a purely scientific resource with as high scientific qualities as possible (I call this the second consensus).

As far as I can see the conflict between certain groups and individuals regard those features and functionality of the trackback system which contradict the second consensus, namely that the arxiv.org should remain purely scientific. It is fair to say – and you confirmed this in your first sentence quoted above – that these non-scientific features of the trackback system such as “intemperate language, sloppy scientific thinking, etc.” are UNAVOIDABLE. It might seem that the only way out is throwing out the whole trackback system and we should say farewell to the advantages of the first consensus for the sake of the second consensus. I would like to argue that this is not the case.

The only problem with the negative effects of the trackback system is that it takes place on arxiv.org. Thus, a promising way out of the current situation would be to move the trackbacks and also open discussion in the form of internet forums to a separate website, say arxiv-forum.org, where it would not pose a problem to the second consensus.

This new site would NOT be integrated into arxiv.org hence all the problems you mentioned (refering to the issues raised by Ginsparg) regarding such an integration does not apply. Perhaps arxiv-forum.org could be loosely associated to arxiv.org similarly to the bittorrent service on http://torrent.rutgers.edu/, but officially and practically it would be a separate site, so those who do not wish to read/write comments, blog entries, etc, would continue using arxiv.org without noticing the change, since there would be NO change concerning arxiv.org. Those who think that an open environment for discussion, blogging, commenting, etc. is desirable would start using the new site parallel to the old one.

There is still of course the question of “who can post comments and/or send trackbacks?”. It came up many times during the current discussion that the usual endorsement system of the arxiv would do the job. A reasonable argument against it was that it would allow too much “intemperate language, sloppy scientific thinking”, personal insults, trolling, etc, but since it would not take place on arxiv.org it would not be in conflict with the second consensus. Thus there sould be one small piece of connection between arxiv.org and arxiv-forum.org, namely the shared endorsement system. Technically this can be solved easily, even without modifying the arxiv.org software.

This setting I believe solves the current problem by separeting the two types of (almost) equally fruitful, enjoyable and useful activity of the community. The more formal, strictly scientific content stays on arxiv.org and the UNAVOIDABLY sloppy, casual, loose, subjective, personal (perhaps insulting) content moves to arxiv-forum.org including the trackback system and open discussion forum. Both activity would find its right place without disturbing the other. The negative sides of the sloppiness will not cause a problem since arxiv-forum.org would be ADMITTEDLY such a site while it would not be open to total crackpots due to the endorsement system. It would be up to the community to maintain a high (or low) standard.

I believe that this arrangement would be beneficial to all. Once it is agreed upon that it is indeed useful for the whole community finding people to run such a service should be straightforward (since only the software has to be maintain, no moderation is needed) for the same reason as there are enough kind and unselfish people voluntarily helping running the arxiv. I am certain quite a few people would be happy to volunteer.

Any feedback would be very welcome and let me say again that if similar other proposals have already been made (perhaps long ago) than please state briefly the outcome of that discussion and direct me to the relevant pointers.

### Re: A proposal

Just to say that I think this is a very good proposal, that would, I think, solve most of the problems mentioned above. An independent (but linked to) website with a discussion forum and trackback system, but no moderation, except the already existing arXiv endorsement system.

Has this proposal been considered by the arXiv committee?

Best,
Vincent

Posted by: Vincent Bouchard on March 13, 2006 4:29 PM | Permalink | Reply to this

Sorry that my post begins with woit but I want to thank the moderators and Distler for this goog job! And I want to say why the current trackback policy is nice, and why it is good to have criteria chosen such, that they exclude woit.

Interestingly, when one asks woit on his blog:

“Why don’t you write a review article.
Simply collect 100 Stringtheory papers and show why the arguments therein are so mathematically week and/or physically misleading. Then make 4 such articles, send them to physical review and then to Arxiv.
This would be easy for you, since you would only lengthen your blog postings with stronger arguments and you would fulfill easily the criterion of an active researcher. It would you take in the standard scientific procedure where even review articles belong to.”

woit deletes that comment. But it wasn’t even a slightest critique in it.

When one goes angry on such a text, then this is, because of his inabillity to formulate critisism on the level of scientific appropriatnes.

Woit wrote, that he cannot write papers with scientific results whitch could falsify some landscape models, because he thinks that this is generally impossible and the arguments in these papers would be often weak which he claims is the reason why he critisise them.

But when asked, if he could write a scientific review article on the whole bunch of weak arguments in stringtheory papers, that is: to discuss 100 papers and write why they are weak and why they have no or a unsure meaning, then woit gets angry and goes over to censoring and deleting.

I therefore think that the case can be made, he has not much scientific to say. Otherwise he could write such a review article. And his lack of publications is due to an unabillity of doing research. He is, as Distler pointed out just “a net-personality”.

That is: Arxiv as a scientific server was right to ban him. Someone who only aims at entertaining the public with critisism and lack of scientific abillity is not the one, a serious researcher who is struggling with a field in crisis might want to comment his papers for public demolition.

And now coming to the moderators and the moderation:

Moderation in forums or Blogs is not a thing where strictly rules can be stated the moderator must obeye to.
There must be always room enough for personal “thinkings” and “judgement” of the moderators, since every case differs in its own way. Human nature has many faces.
Someone who only aims at entertaining the public with critisisms of strugling researchers, who is unable to go along standard scientific procedures and is unable to even write review articles that articulate critisisms in a more formal way, someone should not be in Arxiv.
It is an appropriate choice, to find a criteria that allows woit, when he does publish something sufficiently.

Posted by: Hans on March 9, 2006 6:04 PM | Permalink | Reply to this

“Hans” = Benni

Posted by: Peter Woit on March 9, 2006 6:56 PM | Permalink | Reply to this
Weblog: blogs for industry
Excerpt: Via Uncertain Principles I stumbled upon the story of a fight over a trackback system among physicists. Sean Carroll reviews the first round of bidding: Some time back we learned that arxiv.org, the physics e-print server that has largely superseded...
Tracked: March 10, 2006 1:25 AM

Today (17th March) is the first I’ve heard of this mess. I can’t think what scientific value trackbacks add to the arxiv. By simple cost-benefit analysis they should go immediately.

There was one argument presented in favour: that some papers on the arxiv turn out to be incorrect. It was not explained why a blog post should have a significantly greater probability of being correct. Or indeed what is wrong with the traditional method of educating yourself on the subject and making a calculation.

If anyone seriously thinks that they can improve their judgement about a paper by scanning trackbacks, I fear for their sanity.

Of course there is some substantive physics discussion in two or three blogs, which is even occasionally educational. The reason I would read these two or three is not because of a trackback, but because I know from experience that they have some expertise in the field.

If you want to read physics discussion from experts, you need to decide who the experts are. There is no way for the arxiv administration to make such a decision, ‘active research’ is a very poor surrogate. There is no *need* for them to make a decision either. If you really want to know what X thinks of Y’s paper, just send an email!

Thomas

Posted by: Thomas Dent on March 17, 2006 10:24 AM | Permalink | Reply to this

### the inverse problem

Of course there is some substantive physics discussion in two or three blogs, which is even occasionally educational. The reason I would read these two or three is not because of a trackback, but because I know from experience that they have some expertise in the field.

We’re discussing inverse problems.

If you find a particular blog worthwhile, you may decide to read it regularly. But I doubt very much that you would recall what I (or anyone else) said about some topic 3 years ago.

Conversely, if you are looking at the abstract to a certain paper, and see a trackback (not necessarily a recent one) from John Baez, you may, based on your past experience with John, decide that he probably has something worthwhile to say about the paper. That might dispose you to follow the link to read what he has to say.

Some other trackback, from someone you find less trustworthy, might not be worth your while to follow.

The inverse problem is to let you, the arXiv user, know who has commented on a given paper, so that you can decide whether you want to go read their commentaries.

To make this worthwhile, going forward, we need to maintain a signal/noise ratio at least as high as “so-and-so wrote a paper in response to this one” (something you would discover by following the “cited-by” link on the abstract page).

That’s the point of the current standard. There’s no way that we can guarantee that all trackback links will lead to something worth reading, any more than we can guarantee that all followup papers will be worth reading. The best we can do is maintain a similar level of selectivity, and let you, the arXiv user, do the rest.

Posted by: Jacques Distler on March 17, 2006 1:02 PM | Permalink | PGP Sig | Reply to this

Jacques, it must be tough being a dictator and censor at ArXiv. You have my deepest sympathies!

Posted by: Neubrain on May 19, 2006 10:04 PM | Permalink | Reply to this