Forum: Heterotic Beast – Topic: Bugs

It would be more sensible if it took me to the point where I last read up to (which I presume it knows).

How does Vanilla keep track of what posts you’ve read?

Forum: Instiki – Topic: nlab

One thing that I am pretty sure that slows down a page load is if the page has a lot of wikilinks on it. I don’t know how it checks all the links, but is there some way that that could be speeded up?

Could you compare (by deleting the cached page and reloading ) how long it takes to build a wikilink-heavy page, versus a “normal” one?

I’m particularly interested in the database-lookup times. As I said via email, the WikiReferences model uses a lot of raw SQL queries (which, therefore, do not benefit from ActiveRecord caching). But I am a little skeptical that is the cause of much of a slowdown.

Forum: Heterotic Beast – Topic: Bugs

When I click on a discussion/whatever then it takes me to the top of the page. It would be more sensible if it took me to the point where I last read up to (which I presume it knows).

Forum: Instiki – Topic: nlab

Right, so on the crashing then I’m just watching the new system and waiting to see what it does and how it responds. I’ll keep a hold of all the logs for statistical purposes (not that I’ve any real idea about statistics …).

One thing that I am pretty sure that slows down a page load is if the page has a lot of wikilinks on it. I don’t know how it checks all the links, but is there some way that that could be speeded up? Urs has some pages with loads and loads of links, and there’s talk of having some pages where everything possible is linked (with CSS to lessen the visual impact).

Forum: Instiki – Topic: nlab

Yes, I was thinking that 1Mb was a bit low. When the googlebot hit last night then it was creating a new log file every 20 minutes or so!

Forum: Instiki – Topic: nlab

It keeps 25 files, of size 1MB each.

Both of these numbers are configurable.

Forum: Instiki – Topic: nlab

It’s even possible that it didn’t crash this morning. It was very slow loading the main page so I restarted it. I couldn’t see from the logs anything special, though.

I’m not inured to instiki crashing! I would love to get to the bottom of it, but it’s hard to know what to monitor, and how to monitor it (especially as it may have been my monitoring procedures that contributed to its crashing).

As you say, let’s see how long it can last.

By the way, how many log files does it keep? Does it just keep renumbering them each time it rotates them? If so, I’d better move them out of the log directory each day.

Forum: Instiki – Topic: nlab

I wouldn’t count this morning’s crash as anything special.

You seem to be inured to the idea of Instiki crashing. I am not. It shouldn’t crash, and there’s something wrong if it does. I’m not even convinced that my PassengerPoolIdleTime theory explains the phenomenon. Let’s see if it can go a week without hiccup.

Forum: Instiki – Topic: nlab

I’ve set that one that kills off processes after 20 requests (PassengerMaxRequests), and I’ve set it up to allow 10 concurrent processes (PassengerMaxPoolSize).

I wouldn’t count this morning’s crash as anything special. More likely just teething problems.

Forum: Instiki – Topic: nlab

atom_with_content renders the last 15 updated pages. So it should take approximately 15 times as long as it takes to render 1 page.

Incidentally, the nLab had crashed when I came in this morning…

Ouch. OK. Now I am puzzled.

Is there a process-limit that gets exceeded on your VPS? (Seems unlikely: if there were too many active processes, you wouldn’t be able to login.) Could it be that Passenger is killing off a long-running response, because it thinks Instiki is “inactive”. If so, try setting PassengerPoolIdleTime to 0.

The default (300) clearly won’t work for you if

1. A list request (say) takes more than 300s to complete.
2. That’s the only request received during the 5-minute period in question. You receive, on average, 3 requests/minute, so it’s not inconceivable that you go for long stretches with no requests at all.

Forum: Heterotic Beast – Topic: Bugs

Is the colour of the icon next to the forum title meant to tell me if there’s new stuff there? If so, it’s not always in sync.

Forum: Instiki – Topic: nlab

I was blocking them at the Apache level. That seemed to be what you were complaining about! Maybe I misunderstood.

I can understand list being so big, but atom_with_content is limited to the last … whatever … revisions, so that shouldn’t be so big, should it?

(I think that to get a better picture from the logs then I need to figure out a way to distinguish those that got cached from those that needed a serious operation. But as I’m back on “vanilla” instiki, my hacks to allow this have been taken out.)

Incidentally, the nLab had crashed when I came in this morning, but it’s been running fairly stably all day. So that might just have been first night jitters (or the rampaging googlebot that came through at about midnight).

Forum: Instiki – Topic: nlab

But it didn’t crash, or become otherwise unresponsive. Which is better than what you were doing before your “clean” install.

Some actions, like list, are $O(N)$. With a Wiki your size, they will take a long time to complete¹. If you want prevent people from calling the list action (or other $O(N)$ actions), I suggest blocking/redirecting it at the Apache level.

I assume you know how to use mod_rewrite to good effect.

But it appears that the nLab operates acceptably, even when you allow such actions.

Forum: Instiki – Topic: nlab

Okay, so after a day’s trading at the nLab, here are the closing prices. This is with nothing disabled.

atom_with_content: 192 requests, 176 taking less than 10s to process, 16 taking 50s to process.

list: 6 requests, 2 taking less than 10s to process, 4 taking 5 minutes to process.

Total number of requests: 4223, breakdown of time taken to process (rounded down to nearest 10s):

  0: 4142
  1: 47
  2: 8
  3: 3
  4: 1
  5: 17
  6: 1
  7: 1
  30: 1
  31: 2
  32: 1

So those atom_with_content and list are a major part of the requests that take longer than 30s to deliver.

Forum: Instiki – Topic: Feature Requests

As far as I can tell, updating the application files on a running Rails application (in production mode) has no effect, until the application is restarted. I, honestly, haven’t thought about the bundled Gems, but I expect the answer is the same.

In any case, ruby bundle is pretty fast (of course, that’s because it’s actually superfluous) if the Gemfile hasn’t changed. I suppose you can stat the Gemfile to see whether you actually need to run ruby bundle at all.

But I don’t think that’s your issue…

Forum: Instiki – Topic: Feature Requests

Okay. So simpler just to have separate copies of the code.

I guess what I’m stumbling around with this is some way to make the update cycle a little simpler. At the moment, I update a live instiki installation from golem, and with the ruby bundle step as well, then the update can take a noticeable length of time. What would be better would be to pull the changes to an offline copy on that machine, then update the live versions from the offline copy.

I can easily do that with the bzr stuff, but how would it work with the gems? Perhaps if the live versions weren’t bzr repositories but were more simply synced with the offline version in some fashion (thinking a bit like how rsync works). Hmm, so the workflow would be:

1. Pull the latest instiki revisions to the “dead” code
2. Run ruby bundle on that
3. Copy any updated files from the “dead” version to the live versions, being sure not to clobber anything special
4. Restart instiki on the live versions

would that work, do you think?

Or is this another case of me thinking something is important which really isn’t.

Forum: Instiki – Topic: nlab

I have a script that exports the day’s revisions to a set of bzr repositories (this is to make it easier for people to keep incremental backups of their webs). I haven’t reinstalled it as a cron job yet, and again I can run it manually.

Root has its usual cron jobs (at least, it does now: in the installed image then it doesn’t run them but lets anacron do it, which is fine except that on a server, anacron isn’t running; however, cron just checks for the existence of anacron, not whether or not it is running). But that’s all.

Forum: Instiki – Topic: Feature Requests

Given that instiki can be installed as a gem …

Instiki cannot be installed as a Gem.

There’s an old (~0.10.x) version, which worked as a Gem, and which is probably still floating around (on the internets, nothing ever really disappears). But that was long before my time, and I have not even thought about packaging the current version as a Gem.

Of course, under Passenger, you can run multiple instances of a Rails application (including Instiki), under different subdirectories (or subdomains, if you have virtual hosts enabled).

(At least with Instiki, that would require separate copies of the code, as each instance would have to point to its own database (in config/database.yml). I suppose one could use soft-links astutely, so that there was really only one copy of the source code, shared by these different instances.)

Forum: Instiki – Topic: nlab

Let’s keep it “off” for the time-being; I don’t have any plans to change anything for the next week, at least.

Do you have any other scripts/cron-jobs running?

Forum: Instiki – Topic: Feature Requests

Is the following possible? I tried a couple of things, but don’t know enough to know if what I tried was all that there was. Given that instiki can be installed as a gem, can I install a system-wide version of the code and then run it as a user, with each user having their own separate instiki process, but sharing the code?

Forum: Instiki – Topic: nlab

I’ve now upgraded the nLab server and reinstalled everything. In particular, I pulled a fresh copy of instiki from the repository and made only two changes:

1. Using mysql as the database
2. Disabled the DNSBL spam check

In particular, I’m not mucking about with the logs just yet. We’ll see how this goes for now.

Now, should I enable the daily check on the instiki source code, or shall I keep that off for the time being and update manually?

Forum: Instiki – Topic: Feature Requests

Something that is a “dummy” is not necessarily “dumb”. A “dummy” simply means a fake …

I am familiar with the usage (there’s not a UK/US distinction).

I was making a lame attempt at humour, whilst making the serious point that these CSS classes are used for styling (generated content), and as structural hooks (for converting the \ref{}s into hyperlinks), hence are not “dummy” in either sense.

Forum: Heterotic Beast – Topic: Bugs

This place doesn’t seem to remember me at the moment. *sigh*

Forum: Instiki – Topic: Feature Requests

Ah, another UK/US distinction. Something that is a “dummy” is not necessarily “dumb”. A “dummy” simply means a fake (although it’s not as pejorative), a “stand in”. Had I meant to be rude, I would have said “dumb CSS classes”.

Forum: Heterotic Beast – Topic: Bugs

Hmmm. /users/nnn/posts?monitored=true doesn’t seem to return anything, here on Golem. But it works fine on my test installation.

Both are running in production mode. The test installation uses sqlite3; this one uses mysql. The (admittedly complicated) SQL join seems not to work on the latter.

Hah! Fixed, now. Boolean comparisons are not the same in SQLite3 and MySQL. Finding a syntax that works in both was … umh … fun.

Forum: Heterotic Beast – Topic: Feature Requests

Hmm. I think tagging a post as having been edited, after the grace period, should suffice.

See what you think.

Forum: Instiki – Topic: Feature Requests

…dummy CSS classes.

I bridle, only at referring to these as “dummy.” I thought that the whole mechanism I invented with those classes was very clever. And even more versatile than I had originally envisioned.

Not “dummy” at all….

Forum: Heterotic Beast – Topic: Feature Requests

I was just thinking that if only the post itself was tagged with the edit then it wouldn’t be clear to someone reading the end of the thread that something earlier had been changed. Having it sorted by “updated_at” would fix this, but at too great a cost, I think. The extra line was meant to mitigate this without resorting the posts.

You’re right that the guest user doesn’t have an empty password. The script has to know guest’s password.

Forum: Instiki – Topic: Feature Requests

Yes, it worked!

And to finish the thought, since I’m generating this from a LaTeX source, it can automatically handle adding the dummy CSS classes.

Forum: Instiki – Topic: Feature Requests

Actually, I’m going the other way. I’m going from LaTeX to Instiki. So the source file has \label{foo} and \ref{foo} in it already and in the LaTeX output then it gets converted to “In (2) we saw that …”. So there’s nothing in the source to put as the text in the anchor and something has to be put in its place. Thus my thinking was to subvert the maruku counter-counting to figure out what text should go there. Putting a class .num_enuma on each element of the list makes maruku think that there is a list to be counted there and so it converts the \ref{foo} to the correct hyperlinked text. If you’re using maruku here, this should work:

1. The first item.

2. The second item.

Now we should be able to refer to 1 and 2. I won’t know if that worked until I save the reply, though!

Anyway, it certainly does work on Instiki. I don’t need another list in the CSS because the counters are already produced by the browser working on the list. All I need is for maruku to count the entries in the list, and for that I just need the dummy num_X classes. You’re right that I need one per list, but that shouldn’t be that many on any given page.

(Okay, time to see if that list worked.)