Recent Posts

Subscribe to Recent Posts 325 posts found

posted 2 years ago
Andrew Stacey 118 posts

Forum: Instiki – Topic: nlab

Okay, let’s take http://ncatlab.org/nlab/show/smooth%20infinity-groupoid%20--%20structures which, according to grep, has the order of 409 wikilinks. (Actually, it has 409 hits for the string [[.) On first load, it was already cached and produced the following figures: 169ms first time, 62ms second. I got 0.026s for “time to first byte” and 0.663s for “from click to complete display”.

Now I delete it from the cache, and try again. A cup-of-tea later, and I get the following: 74074ms (View 72773, DB: 1276). On the receiving end, I get 82s and 87s for the delivery times. Second time, similar figures.

The time this gets a bit annoying is when editing a page, since then it has to regenerate it each time. That’s a fair wait if you’ve only changed a couple of spelling mistakes.

 
posted 2 years ago
distler 77 posts

edited 2 years ago

Forum: Heterotic Beast – Topic: Bugs

Layer 1

It would be more sensible if it took me to the point where I last read up to (which I presume it knows).

How does Vanilla keep track of what posts you’ve read?

 
posted 2 years ago
distler 77 posts

Forum: Instiki – Topic: nlab

One thing that I am pretty sure that slows down a page load is if the page has a lot of wikilinks on it. I don’t know how it checks all the links, but is there some way that that could be speeded up?

Could you compare (by deleting the cached page and reloading ) how long it takes to build a wikilink-heavy page, versus a “normal” one?

I’m particularly interested in the database-lookup times. As I said via email, the WikiReferences model uses a lot of raw SQL queries (which, therefore, do not benefit from ActiveRecord caching). But I am a little skeptical that is the cause of much of a slowdown.

 
posted 2 years ago
Andrew Stacey 118 posts

Forum: Heterotic Beast – Topic: Bugs

When I click on a discussion/whatever then it takes me to the top of the page. It would be more sensible if it took me to the point where I last read up to (which I presume it knows).

 
posted 2 years ago
Andrew Stacey 118 posts

edited 2 years ago

Forum: Instiki – Topic: nlab

Right, so on the crashing then I’m just watching the new system and waiting to see what it does and how it responds. I’ll keep a hold of all the logs for statistical purposes (not that I’ve any real idea about statistics …).

One thing that I am pretty sure that slows down a page load is if the page has a lot of wikilinks on it. I don’t know how it checks all the links, but is there some way that that could be speeded up? Urs has some pages with loads and loads of links, and there’s talk of having some pages where everything possible is linked (with CSS to lessen the visual impact).

 
posted 2 years ago
Andrew Stacey 118 posts

edited 2 years ago

Forum: Instiki – Topic: nlab

Yes, I was thinking that 1Mb was a bit low. When the googlebot hit last night then it was creating a new log file every 20 minutes or so!

 
posted 2 years ago
distler 77 posts

edited 2 years ago

Forum: Instiki – Topic: nlab

It keeps 25 files, of size 1MB each.

Both of these numbers are configurable.

 
posted 2 years ago
Andrew Stacey 118 posts

edited 2 years ago

Forum: Instiki – Topic: nlab

It’s even possible that it didn’t crash this morning. It was very slow loading the main page so I restarted it. I couldn’t see from the logs anything special, though.

I’m not inured to instiki crashing! I would love to get to the bottom of it, but it’s hard to know what to monitor, and how to monitor it (especially as it may have been my monitoring procedures that contributed to its crashing).

As you say, let’s see how long it can last.

By the way, how many log files does it keep? Does it just keep renumbering them each time it rotates them? If so, I’d better move them out of the log directory each day.

 
posted 2 years ago
distler 77 posts

edited 2 years ago

Forum: Instiki – Topic: nlab

I wouldn’t count this morning’s crash as anything special.

You seem to be inured to the idea of Instiki crashing. I am not. It shouldn’t crash, and there’s something wrong if it does. I’m not even convinced that my PassengerPoolIdleTime theory explains the phenomenon. Let’s see if it can go a week without hiccup.

 
posted 2 years ago
Andrew Stacey 118 posts

edited 2 years ago

Forum: Instiki – Topic: nlab

I’ve set that one that kills off processes after 20 requests (PassengerMaxRequests), and I’ve set it up to allow 10 concurrent processes (PassengerMaxPoolSize).

I wouldn’t count this morning’s crash as anything special. More likely just teething problems.

 
posted 2 years ago
admin 57 posts

edited 2 years ago

Forum: Instiki – Topic: nlab

atom_with_content renders the last 15 updated pages. So it should take approximately 15 times as long as it takes to render 1 page.

Incidentally, the nLab had crashed when I came in this morning…

Ouch. OK. Now I am puzzled.

Is there a process-limit that gets exceeded on your VPS? (Seems unlikely: if there were too many active processes, you wouldn’t be able to login.) Could it be that Passenger is killing off a long-running response, because it thinks Instiki is “inactive”. If so, try setting PassengerPoolIdleTime to 0.

The default (300) clearly won’t work for you if

  1. A list request (say) takes more than 300s to complete.
  2. That’s the only request received during the 5-minute period in question. You receive, on average, 3 requests/minute, so it’s not inconceivable that you go for long stretches with no requests at all.
 
posted 2 years ago
Andrew Stacey 118 posts

Forum: Heterotic Beast – Topic: Bugs

Is the colour of the icon next to the forum title meant to tell me if there’s new stuff there? If so, it’s not always in sync.

 
posted 2 years ago
Andrew Stacey 118 posts

edited 2 years ago

Forum: Instiki – Topic: nlab

I was blocking them at the Apache level. That seemed to be what you were complaining about! Maybe I misunderstood.

I can understand list being so big, but atom_with_content is limited to the last … whatever … revisions, so that shouldn’t be so big, should it?

(I think that to get a better picture from the logs then I need to figure out a way to distinguish those that got cached from those that needed a serious operation. But as I’m back on “vanilla” instiki, my hacks to allow this have been taken out.)

Incidentally, the nLab had crashed when I came in this morning, but it’s been running fairly stably all day. So that might just have been first night jitters (or the rampaging googlebot that came through at about midnight).

 
posted 2 years ago
admin 57 posts

edited 2 years ago

Forum: Instiki – Topic: nlab

But it didn’t crash, or become otherwise unresponsive. Which is better than what you were doing before your “clean” install.

Some actions, like list, are O(N). With a Wiki your size, they will take a long time to complete1. If you want prevent people from calling the list action (or other O(N) actions), I suggest blocking/redirecting it at the Apache level.

I assume you know how to use mod_rewrite to good effect.

But it appears that the nLab operates acceptably, even when you allow such actions.


  1. Incorporating will_paginate (which is used by Heterotic Beast, for just this reason) in those actions will make them O(1), again.

 
posted 2 years ago
Andrew Stacey 118 posts

edited 2 years ago

Forum: Instiki – Topic: nlab

Okay, so after a day’s trading at the nLab, here are the closing prices. This is with nothing disabled.

atom_with_content: 192 requests, 176 taking less than 10s to process, 16 taking 50s to process.

list: 6 requests, 2 taking less than 10s to process, 4 taking 5 minutes to process.

Total number of requests: 4223, breakdown of time taken to process (rounded down to nearest 10s):

  0: 4142
  1: 47
  2: 8
  3: 3
  4: 1
  5: 17
  6: 1
  7: 1
  30: 1
  31: 2
  32: 1

So those atom_with_content and list are a major part of the requests that take longer than 30s to deliver.

 
posted 2 years ago
distler 77 posts

Forum: Instiki – Topic: Feature Requests

As far as I can tell, updating the application files on a running Rails application (in production mode) has no effect, until the application is restarted. I, honestly, haven’t thought about the bundled Gems, but I expect the answer is the same.

In any case, ruby bundle is pretty fast (of course, that’s because it’s actually superfluous) if the Gemfile hasn’t changed. I suppose you can stat the Gemfile to see whether you actually need to run ruby bundle at all.

But I don’t think that’s your issue…

 
posted 2 years ago
Andrew Stacey 118 posts

Forum: Instiki – Topic: Feature Requests

Okay. So simpler just to have separate copies of the code.

I guess what I’m stumbling around with this is some way to make the update cycle a little simpler. At the moment, I update a live instiki installation from golem, and with the ruby bundle step as well, then the update can take a noticeable length of time. What would be better would be to pull the changes to an offline copy on that machine, then update the live versions from the offline copy.

I can easily do that with the bzr stuff, but how would it work with the gems? Perhaps if the live versions weren’t bzr repositories but were more simply synced with the offline version in some fashion (thinking a bit like how rsync works). Hmm, so the workflow would be:

  1. Pull the latest instiki revisions to the “dead” code
  2. Run ruby bundle on that
  3. Copy any updated files from the “dead” version to the live versions, being sure not to clobber anything special
  4. Restart instiki on the live versions

would that work, do you think?

Or is this another case of me thinking something is important which really isn’t.

 
posted 2 years ago
Andrew Stacey 118 posts

edited 2 years ago

Forum: Instiki – Topic: nlab

I have a script that exports the day’s revisions to a set of bzr repositories (this is to make it easier for people to keep incremental backups of their webs). I haven’t reinstalled it as a cron job yet, and again I can run it manually.

Root has its usual cron jobs (at least, it does now: in the installed image then it doesn’t run them but lets anacron do it, which is fine except that on a server, anacron isn’t running; however, cron just checks for the existence of anacron, not whether or not it is running). But that’s all.

 
posted 2 years ago
distler 77 posts

edited 2 years ago

Forum: Instiki – Topic: Feature Requests

Given that instiki can be installed as a gem …

Instiki cannot be installed as a Gem.

There’s an old (~0.10.x) version, which worked as a Gem, and which is probably still floating around (on the internets, nothing ever really disappears). But that was long before my time, and I have not even thought about packaging the current version as a Gem.

Of course, under Passenger, you can run multiple instances of a Rails application (including Instiki), under different subdirectories (or subdomains, if you have virtual hosts enabled).

(At least with Instiki, that would require separate copies of the code, as each instance would have to point to its own database (in config/database.yml). I suppose one could use soft-links astutely, so that there was really only one copy of the source code, shared by these different instances.)

 
posted 2 years ago
distler 77 posts

edited 2 years ago

Forum: Instiki – Topic: nlab

Let’s keep it “off” for the time-being; I don’t have any plans to change anything for the next week, at least.

Do you have any other scripts/cron-jobs running?

 
posted 2 years ago
Andrew Stacey 118 posts

Forum: Instiki – Topic: Feature Requests

Is the following possible? I tried a couple of things, but don’t know enough to know if what I tried was all that there was. Given that instiki can be installed as a gem, can I install a system-wide version of the code and then run it as a user, with each user having their own separate instiki process, but sharing the code?

 
posted 2 years ago
Andrew Stacey 118 posts

edited 2 years ago

Forum: Instiki – Topic: nlab

I’ve now upgraded the nLab server and reinstalled everything. In particular, I pulled a fresh copy of instiki from the repository and made only two changes:

  1. Using mysql as the database
  2. Disabled the DNSBL spam check

In particular, I’m not mucking about with the logs just yet. We’ll see how this goes for now.

Now, should I enable the daily check on the instiki source code, or shall I keep that off for the time being and update manually?

 
posted 2 years ago
distler 77 posts

Forum: Instiki – Topic: Feature Requests

Something that is a “dummy” is not necessarily “dumb”. A “dummy” simply means a fake …

I am familiar with the usage (there’s not a UK/US distinction).

I was making a lame attempt at humour, whilst making the serious point that these CSS classes are used for styling (generated content), and as structural hooks (for converting the \ref{}s into hyperlinks), hence are not “dummy” in either sense.

 
posted 2 years ago
Andrew Stacey 118 posts

Forum: Heterotic Beast – Topic: Bugs

This place doesn’t seem to remember me at the moment. *sigh*

 
posted 2 years ago
Andrew Stacey 118 posts

Forum: Instiki – Topic: Feature Requests

Ah, another UK/US distinction. Something that is a “dummy” is not necessarily “dumb”. A “dummy” simply means a fake (although it’s not as pejorative), a “stand in”. Had I meant to be rude, I would have said “dumb CSS classes”.

 
posted 2 years ago
distler 77 posts

edited 2 years ago

Forum: Heterotic Beast – Topic: Bugs

Hmmm. /users/nnn/posts?monitored=true doesn’t seem to return anything, here on Golem. But it works fine on my test installation.

Both are running in production mode. The test installation uses sqlite3; this one uses mysql. The (admittedly complicated) SQL join seems not to work on the latter.

Hah! Fixed, now. Boolean comparisons are not the same in SQLite3 and MySQL. Finding a syntax that works in both was … umh … fun.

 
posted 2 years ago
distler 77 posts

Forum: Heterotic Beast – Topic: Feature Requests

Hmm. I think tagging a post as having been edited, after the grace period, should suffice.

See what you think.

 
posted 2 years ago
distler 77 posts

Forum: Instiki – Topic: Feature Requests

…dummy CSS classes.

I bridle, only at referring to these as “dummy.” I thought that the whole mechanism I invented with those classes was very clever. And even more versatile than I had originally envisioned.

Not “dummy” at all….

 
posted 2 years ago
Andrew Stacey 118 posts

Forum: Heterotic Beast – Topic: Feature Requests

I was just thinking that if only the post itself was tagged with the edit then it wouldn’t be clear to someone reading the end of the thread that something earlier had been changed. Having it sorted by “updated_at” would fix this, but at too great a cost, I think. The extra line was meant to mitigate this without resorting the posts.

You’re right that the guest user doesn’t have an empty password. The script has to know guest’s password.

 
posted 2 years ago
Andrew Stacey 118 posts

Forum: Instiki – Topic: Feature Requests

Yes, it worked!

And to finish the thought, since I’m generating this from a LaTeX source, it can automatically handle adding the dummy CSS classes.