Skip to the Main Content

Note:These pages make extensive use of the latest XHTML and CSS Standards. They ought to look great in any standards-compliant modern browser. Unfortunately, they will probably look horrible in older browsers, like Netscape 4.x and IE 4.x. Moreover, many posts use MathML, which is, currently only supported in Mozilla. My best suggestion (and you will thank me when surfing an ever-increasing number of sites on the web which have been crafted to use the new standards) is to upgrade to the latest version of your browser. If that's not possible, consider moving to the Standards-compliant and open-source Mozilla browser.

February 16, 2005

Coming Soon to a Hard Drive Near You

Once upon a time, a megabyte was a lot of data. In 1989, when Joanne Cohn first started emailing preprints to a couple of hundred colleagues, people quickly found themselves exceeding their mail quotas. And not everyone was interested in every paper. So why waste bandwidth and precious disk space on all that junk?

The idea of centralizing the storage, and sending people only the papers they requested, prompted Paul Ginsparg to start the hep-th archive in 1991.

Flash forward 14 years.

250 GB hard drives are cheap, and a laptop with less than 60 GB seems positively claustrophobic. Hep-th has grown tremendously. But even with over two hundred submissions a month, it’s still a puny amount of data by today’s standards. The entire archive from 1991 to the present (8GB of PDF files) fits easily on an iPod.

So, in reversal of history, Joanna Karczmarek is gearing up to distribute the whole shebang via bittorent. She’s currently offering 2004 (pdf papers and a plain-text list of abstracts) as a modest 850 MB torrent.

A year’s worth of physics for your iPod Shuffle.

Posted by distler at February 16, 2005 3:27 PM

TrackBack URL for this Entry:   https://golem.ph.utexas.edu/cgi-bin/MT-3.0/dxy-tb.fcgi/513

5 Comments & 1 Trackback

Re: Coming Soon to a Hard Drive Near You

Perhaps a couple instructions for bittorrent unix users.

Of course, get bittorrent. apt-get install bittorrent; apt-get install bittorrent-gui, etc…

get the two .torrent files from Joanna. Put them in some directory. Open a terminal. Then you have two options:

For single .torrent download
btdownloadcurses this.torrent

For multiple torrents:
btlaunchmanycurses .
(note the dot, that tells the interface to look for .torrent data in the current directory)


Posted by: Alejandro Rivero on February 17, 2005 5:25 AM | Permalink | Reply to this

Re: Coming Soon to a Hard Drive Near You

Now I see it, Joanna has also some instructions for unix

She suggests as a first test to try btdownloadheadless.py –url http://cartan.harvard.edu/index.torrent –saveas /home/myusername/hep-th/index.txt

It is a regret that IPod can not still to do ebooks. If you are evil enough with the electronics, you can be interested on the Gumstix. You will need a basic module ($109) plus a LCD module ($27) plus a Compact Flash module ($25) plus your preferred LCD display :-D…

Posted by: Alejandro Rivero on February 17, 2005 11:48 AM | Permalink | Reply to this

Re: Coming Soon to a Hard Drive Near You

I have whished for something like that for a while now, but was too afraid of Paul Ginsparg’s wrath to try to wget it all :-)

Now we would only need a rsync-kind of way to get updates. It would be good enough if the updated torrent file would only pull the changed PDFs (those with a different checksum). At least some Bittorrent clients can do this, but one needs to seed the individual files and not tar archives.

Posted by: Volker Braun on February 17, 2005 4:05 PM | Permalink | Reply to this

Re: Coming Soon to a Hard Drive Near You

Volker, as I can tell from experience, you were right.

Lets see if Joanna idea works and then we would think of to bittorrent partial updates.

Posted by: Alejandro Rivero on February 18, 2005 4:37 AM | Permalink | Reply to this

Re: Coming Soon to a Hard Drive Near You

That is a wonderful idea! one useful feature is better search abilities if you have the files locally. For example Acrobat will let you search for specifc word combination within the paper. One can envision some software that shows you all the hyperlink tree emanating from a specific paper, or many other useful features to automate the way you use this database.

Posted by: Moshe Rozali on February 20, 2005 9:16 AM | Permalink | Reply to this
Read the post The arXiv in your pocket.
Weblog: The String Coffee Table
Excerpt: As pointed out on several other blogs, Joanna Karczmarek has been testing the waters with a downloadable version of the...
Tracked: April 12, 2005 11:53 PM

Post a New Comment