# Recent Posts by distler

105 posts found

Forum: Instiki – Topic: Bugs

If you’ve found a mistake in the rake task, please send me a patch.

Forum: Heterotic Beast – Topic: MySQL Gotcha

If you’re going to use Heterotic Beast in production, you need to be running MySQL 5.5.3 or later, and follow the advice in this blog post.

Otherwise, the lack of support for Unicode will come back to bite you.

Forum: itex2MML – Topic: weird math fonts

P.S.: Congratulations on figuring out how to make this page ill-formed! It took a bit of work to fix the issue.

Forum: Instiki – Topic: Some questions

• Is there a way to link to a category? My idea is to have a lot of categories, but only link to the most important ones from the homepage. I couldn’t figure out any way to do that.

Not sure what you are after. Perhaps you mean to link to the page listing all pages in category ‘foo’. The url for that is /list/foo .

• Macros for iTex: Is there any way to define per-page or global macros?

No. Though this is a much-discussed question.

• Errors for iTex: So far it seems like if some TeX expression doesn’t work you just get the source rendered, but there is no way to find exactly where the error is. This way if I have a long equation I am left to hunt through the whole thing for the missed bracket or parenthesis. Is there something I’m missing?

I agree that itex’s error-reporting is pretty useless. Depending on the type of error (a missed brace bracket, say), LaTeX’s is often not much better. Here, at least, you know which equation to look at for the error, as each equation is parsed separately, and errors can’t spill over as they sometimes do in LaTeX.

• Linking and/or embedding local files: I am running Instiki locally, but I am syncing the whole thing online, so I can use it from more than one computer. I often use Xournal (on a tablet PC) to take notes/do calculations. While for high-level results, or summaries, Instiki is fine, for long/messy calculations it’s a lot faster to just hand-write them in Xournal. Ideally, I want to be able to link to a Xournal file from Instiki and have some quick way of viewing or editing it. Right now it seems like that the only way is to put a file:/// url, but that requires syncing two things separately, and making sure the url’s make sense on every computer I am using.

Look at Instiki’s file upload capability.

That probably doesn’t help you very much from the point of view of syncing between different computers (as each Instiki installation will have its own set of uploaded files).

• Editing SVG graphics: This is something that I’m pretty sure is a bug. It seems like unless there is empty space before and after the svg tags, the “Edit SVG graphic” button doesn’t show up.

I think it doesn’t like ”<svg” as the first characters on a page. But I have not had any trouble if the graphic is in the middle of the text.

• How can I adjust the way math is rendered? I am using Firefox 11 under Linux. All math is much smaller than the surrounding text (e.g. 0 is the height of a lower case letter). I imagine there is some CSS option to change the font size for math, but I couldn’t figure out what it was.

That’s strange. I am using Firefox on a Mac, and see no such inconsistency. Do you have the STIX fonts installed?

• I have some programing experience, though I haven’t used Ruby before. I am willing to try to add some of these features myself, if you can give me some pointers as to where to start.

Contributions are always welcome. The source repository is available both through BZR and on Github.

Why don’t you click on the “Source” link, say, at http://golem.ph.utexas.edu/wiki/instiki/show/HomePage, to see how it’s done. Of course, you also want to look at the source of the Sidebar itself.

Forum: Instiki – Topic: S5 presentation export

It’s not so clear to me what the best approach is.

The number of CSS and Javascript files (MathJax, in particular, is huge) required to support a single S5 slideshow is very large.

It would be possible, but very inefficient, to produce a Zip file, with all of that junk, for every slideshow you decided to export.

In some ways, you’d be better off putting an entire Instiki installation on a USB stick (assuming that the host computer has Ruby).

A hybrid approach would be to export a static file of the S5 slideshow, with the URLs rewritten to point to the files in an Instiki installation.

Gotta think some more …

Forum: Instiki – Topic: Feature Requests

I’m sure there’s a solution for that: …

Is this a feature that is implemented somewhere?

Your short description is slightly … underspecified. So looking at an actual implementation would be helpful to me, in deciding whether this is something to implement in Instiki.

Forum: Instiki – Topic: nlab

Is it obvious why the spider aren’t just hitting the cache (in which case, they should not slow down the system at all)?

Are they asking for all revisions of some page (or whatever), that would entail a large percentage of cache-misses?

I ask, just because it seems to me that, if they are operating correctly, spiders shouldn’t lead to an undue slowdown. Maybe I’ve been remiss about

<meta name="robots" content="noindex,nofollow" />

directives.

In any case, is it clear that your 3-queue scheme is better than having one queue, with a larger number of worker processes? (I.e., do these spiders insist on making multiple simultaneous connections, or do they access the nlab serially?)

Forum: Instiki – Topic: Bugs

That’s pretty strange. I have not been able to reproduce the problem.

Obviously, having dummy /app/helper/admin_helper.rb and /app/helper/file_helper.rb files does no harm, but I’m puzzled by the fact that they seem to be necessary for you.

Forum: Instiki – Topic: Bugs

Ah. I see.

Forum: Instiki – Topic: Bugs

Sorry.

Maruku’s HTML output looks perfectly OK (ie, as expected) in your example(s). (View the XHTML source, and tell me what you think is wrong with it.)

What you are complaining about is that the CSS doesn’t produce the desired appearance of the output.

Since you’ve been mucking about with the CSS of the Theorem Environment, why don’t you muck about some more and get it to format your example correctly.

Forum: Instiki – Topic: Bugs

Are you using the current revision of Instiki? ‘Cuz that’s not the behaviour I see, when I follow the steps above.

Forum: Instiki – Topic: Bugs

I’m not so sure I understand.

1. I created a page, “Foo”.
2. I edited the page, “Foo”, and changed the name to “Bar”.
3. I removed the [[!redirects Foo]] directive, and saved the page.

All instances of “Foo” were successfully removed from the Cache.

What am I missing?

Forum: Instiki – Topic: nlab

Given the number of links that Urs has on some of the nLab pages…

It’s the number of inbound links that matters, but yeah.

If the page name doesn’t change, surely then you don’t have to expire any of the pages that refer to it?

You do, for a newly-created page… but not, I agree, for a revision of an existing page. I was, somewhat crudely, not distinguishing between those cases. It occurs to me that I can use an after_create hook to distinguish between those cases.

Forum: Instiki – Topic: nlab

It seems to be the list and recently_revised ones that get expired several times in a row.

That would be a consequence of

1. When a page is saved, expire all pages that reference that page.
2. When you expire a page, also expire the corresponding “index pages’ (list, recently-revised, atom feeds).

The first is further-complicated by the facility for renaming pages. That means we need to expire all the pages that refer to the old page and all the pages that refer to the new page.

I guess that could be optimized better for the case where the page doesn’t change names, as we don’t have to expire the same pages twice. I think the current procedure was motivated by complaints (from y’all) that, in some circumstances, pages were not being expired when they should.

Forum: Instiki – Topic: nlab

On my machine, each Expired fragment takes 0.1-0.2 ms. So to get 10-20 s, you’re talking about expiring $~{10}^{5}$ fragments?!

Wow! That’s impressive.

Forum: Instiki – Topic: Bugs

I’m a little baffled. It should work. Sometimes it does (saving the file triggers the cache sweeper); sometimes it doesn’t. I can’t see what the difference is.

Will have to investigate further …

Forum: Instiki – Topic: Feature Requests

I feel that there is a qualitative difference between a wikilink and a hyperlink

Indeed, there is. And that’s why there are no anchors allowed.

If page “Foo” has multiple WikiLinks to [[Bar]], there’s still only one corresponding entry in the database. That could no longer be true if “Foo” could link to [[Bar]] and to [[Bar#baz]].

What you want is a hyperlink, and Markdown provide a syntax for hyperlinks which permits anchors.

### Update:

On reflection, I suppose that allowing the presence of anchors doesn’t strictly conflict with having just one entry in the database. I should think some more …

Forum: Instiki – Topic: Bugs

Because of that last, my guess is that the pre-upload version is still in the cache, but that the page shown when the file is uploaded doesn’t read the cache version.

You’re probably correct. The rule is that pages with Flash messages on them (like the one that tells you that the file was successfully-uploaded) are not cached. So you get to see the correct page once, but if the incorrect one wasn’t deleted from the cache, that’s what you’ll see the second time.

Fixed in Revision 756 Revision 757.

Forum: Heterotic Beast – Topic: Rails 3.1.0

I upgraded Heterotic Beast to Rails 3.1.0. Despite all my prior testing, the process didn’t go as smoothly as I would have liked, and this forum was pretty disrupted for most of Friday.

Should be back to normal now. But leave a comment here, if something’s still broken for you.

The main new feature is the Asset pipeline, which supposedly speeds the delivery of static files (CSS, javascript, and images). Unfortunately, the result seems buggy.

• The reference

"#{asset_path('something.png')}"

sometimes turns into (the correct)

"/forum/assets/something-5c4374aa4b1911ebbabb73883b3cd5c0.png"

and sometimes it turns into (the incorrect)

"/assets/something-5c4374aa4b1911ebbabb73883b3cd5c0.png"

I’m using some Apache-fu to redirect the latter, but that shouldn’t be necessary.

• I had to switch to Sass (from .css.erb) to get URLs for background images to include the fingerprint. I.e., within a .css.erb file, the above generates

"/assets/something.png"

Other minor bugs include:

• The acts_as_state_machine gem uses some deprecated methods, which generate a warning in the User model. There’s a Rails 3.1 fork which fixes the problem. But it’s unclear when, if ever, that will be released as a gem.

• Prototype 1.7.0 generates a Javascript error

 Error: mismatched tag. Expected </link>.
Source File:
<div xmlns="http://www.w3.org/1999/xhtml"><link></div>

The problem is in the LINK_ELEMENT_INNERHTML_BUGGY function, where you could replace <link> with <link/> to silence the error. This was not a problem in Prototype 1.6.x.

Forum: Instiki – Topic: Bugs

Of course they are different.

• <a> </a>” is an ”a” element with a single child node, which is a text node, containing ” “.
• <a></a>” is an ”a” element with no children.

In X(HT)ML, the latter is equivalent to ”<a/>”.

The short-tag construction does not exist in HTML and all browsers interpret the latter as the opening tag, ”<a>”, of an ”a” element (which is not what you want).

Forum: Instiki – Topic: Bugs

<a id="anchor"> </a>

Here's a [[wikilink]

would not have triggered the bug. Only empty elements (which get converted to short-tag syntax, <a id="anchor"/>, in the output) triggered this bug. Since you probably don’t want empty a or code elements (they are perfectly correct in XHTML, but wreak havoc, when the same document is parsed as HTML), you probably didn’t want the problematic (if you prefer that to uselesss) empty elements in the first place.

Forum: Instiki – Topic: Bugs

Same thing happen(ed) when you typed (the equally useless)

<code></code>

Here's a [[wikilink]].

Fixed in Revision 744.

Forum: Heterotic Beast – Topic: Bugs

No it doesn’t.

Deleting the only post in a topic, also deletes the topic. Possibly, the redirect (which normally goes to topic#show but, in this case, should go to the forum#show because the topic no longer exists) is incorrect.

Forum: Instiki – Topic: nlab

Thinking about Recently Revised and All Pages, you suggested (somewhere) taking them out of the sweeper as a way of stopping them being regenerated every time a page is edited (I don’t know if this was one of you “If you’re going to do something crazy, here’s a way of limiting how crazy you’re going to be” suggestions or if you thought this was actually a good idea).

The former. You’re trading off workload on the server for stale data. Since computers are supposed to serve humans, rather than the other way around, the question is: does this improve the user experience?

Say you implement the above suggestion. On the one hand, the user always (or almost always, depending on implementation) receives the cached page, i.e. gets a quick response. On the other hand, the data is invariably stale.

Leaving these alone, the user is guaranteed to receive fresh data, but there could be a significant delay, if the page has to be regenerated. What percentage of requests for these pages hit the cache?

A better solution is to pull in the will_paginate gem, and paginate the data returned. That makes the request $O\left(1\right)$, again, instead of $O\left(N\right)$. So the user gets both a quick response AND up-to-date data.

As to moving away from Maruku, to some peg-markdown-based clone, note that

1. This will benefit Heterotic Beast as well.
2. The task I’m asking of your guys is not “programming,” per se. It involves writing a formal PEG grammar for Maruku’s extended Markdown syntax (starting with the existing PEG grammar, already in peg-markdown).
3. On the other hand, if there were someone handy in C, that would be most appreciated, too, because I am crappy at C. Hooking in itex2MML would take mere minutes for a competent C-programmer.
4. I imagine that a such a fork of peg-markdown (as it’s written in C), would be useful in your other projects, as well.

Forum: Instiki – Topic: nlab

Did a lot of profiling this weekend, and produced a few tweaks to Maruku’s parsing, which speeded it up a little.

Unfortunately, the main discovery was that (with that test page as input), 3/4 of Maruku’s time is spent in the #to_html output method; only 1/4 is spent in parsing the original input. Thus, my efforts, which maybe improved the parsing speed by 5%, contributed at best a 1% speedup in the total Instiki processing time, i.e something you would never notice.

I hope that one of your guys finds formal grammars sufficiently “categorical” to be worthy of a small bit of their attention.

Forum: Instiki – Topic: nlab

Pretty much everything beyond the standard Markdown syntax needs to be written, though some folks in the peg-markdown network seem to have included Michel Fortin’s Markdown-Extra extensions (albeit, along with some of their own, incompatible, extensions).

Presumably, this fork of peg-markdown would be directly linked to the itex2MML functions which process inline and display equations (ie, it would not use the Ruby bindings provided by the itextomml gem). So there’s a little bit more to do than write the peg grammar. But not a lot more …

As to whether you want to ask someone to do this, I’ve already explained my desire to replace Maruku (for licensing reasons). Here’s another motivation, from efficiency. Instiki’s performance does genuinely suck, in this instance.

The question is, are they (your nlab colleagues) willing to do anything about improving it?

Feel free to point them to this discussion, and to the previous one on Markdown alternatives.

Forum: Instiki – Topic: nlab

On golem (the iMac on my desk), rendering a copy of that page takes

Completed in 27588ms (View: 27356, DB: 192)

Note that it does spend (what I consider to be) a significant amount of time querying the database, but it is totally dwarfed by the rendering time. (I don’t know why yours is spending an order of magnitude longer querying the database. Seems like something’s very wrong, there, even though the conclusion is the same.)

The page, of course, contains a number of markup errors, like

... [[transversal map]s ...

which sent Maruku into convulsions. Surprisingly, correcting those errors did not appreciably affect the rendering time (the above-reported time is after making the relevant markup corrections).

On my laptop, a typical time was

Completed in 48775ms (View: 48635, DB: 77)

SQlite3 is faster than MySQL, but the machine itself is significantly slower than the iMac.

Of those 49 seconds, spent rendering the page, 43 of them were spent in Maruku (for obscure reasons, Maruku has to be run twice, so really, we’re talking about 22 seconds to process the 175KB source).

Maruku doesn’t particularly care about the number of WikiLinks, so that has nothing to do with why it takes so long render this particular page.

Of the remaining 6 seconds, 4 seconds were spent in the Instiki Sanitizer. I don’t think there’s much to optimize there.

The remaining 2 seconds were, largely, spent in the Chunk-Handler – the thing that processes Wikilinks (and, presumably, cares about how many of them there are). 2 seconds is still a long time, but it’s not surprising. Doing on the order of ${10}^{3}$ RegExp substitutions (5360 chunk-masking and 686 chunk-unmasking operations, to be precise) on a 175KB string, takes significant time. Using Regexps to process long strings sucks.

I have looked at various optimizations of the Chunk-Handler code, but nothing I can do will contribute much to the speedup of rendering this page, which is dominated by Maruku.

Now, if one of your nLab folks were to volunteer to write a PEG grammar for Maruku’s extended Markdown syntax, …

#### Update:

Now, if one of your nLab folks were to volunteer to write a PEG grammar for Maruku’s extended Markdown syntax, …

Since I’m not gonna hold my breath for that to happen, I decided to spend some time (alas, more than I expected) making Maruku faster. The new rendering times for that page are

Completed in 13228ms (View: 13024, DB: 198)

on golem and

Completed in 21666ms (View: 20979, DB: 83)

on my laptop. Roughly a factor of 2 improvement in the total rendering time, in both cases.

Still not great, but it’s the best that I am going to achieve.

Forum: Heterotic Beast – Topic: Bugs

It would be more sensible if it took me to the point where I last read up to (which I presume it knows).

How does Vanilla keep track of what posts you’ve read?

Forum: Instiki – Topic: nlab

One thing that I am pretty sure that slows down a page load is if the page has a lot of wikilinks on it. I don’t know how it checks all the links, but is there some way that that could be speeded up?

Could you compare (by deleting the cached page and reloading ) how long it takes to build a wikilink-heavy page, versus a “normal” one?

I’m particularly interested in the database-lookup times. As I said via email, the WikiReferences model uses a lot of raw SQL queries (which, therefore, do not benefit from ActiveRecord caching). But I am a little skeptical that is the cause of much of a slowdown.