Recent Posts

Subscribe to Recent Posts 455 posts found

posted 9 years ago
tanzer 36 posts

Forum: Instiki – Topic: How do I bulk-clear the Instiki cache

I did a bulk update in the database to change our forum URL from one location to another. (Updated the field Content in the Revisions table.)

Now a large number of entries in the cache are stale.

I grepped in the cache directory tree for the old url, and found it present in files in several subdirectories of the cache/views folder for the web in question. I saw it in the subdirectories source, tex, revision, …

In addition, the old URL is present in files in the top-level folder “storage.”

What is the cleanest / simplest / safest way to purge the entire cache?

Thanks!

 
posted 9 years ago
tanzer 36 posts

Forum: Instiki – Topic: Idle Instiki was using 17% of CPU on a shared host

Good questions. Thanks. Research pending…

 
posted 9 years ago
distler 123 posts

Forum: Instiki – Topic: Idle Instiki was using 17% of CPU on a shared host

Hmmm. Both of those sound bad. Are they related?

What version of Passenger are you running?

 
posted 9 years ago
tanzer 36 posts

Forum: Instiki – Topic: Idle Instiki was using 17% of CPU on a shared host

But this isn’t my main concern tonight, which is off-topic: our shared host is crashing everyday, due to too many sleeping processes and not enough memory. They claim that 5G is enough memory, and are promising to fix it… Grr.

 
posted 9 years ago
tanzer 36 posts

Forum: Instiki – Topic: Idle Instiki was using 17% of CPU on a shared host

Noticed this, then I touched tmp/restart.txt and usage went down to basically zero.

 
posted 9 years ago
tanzer 36 posts

Forum: Instiki – Topic: Feature Requests

Thanks!

 
posted 9 years ago
distler 123 posts

Forum: Instiki – Topic: Feature Requests

Try adding the entry

'dnsbl.tornevall.org' => 'https://dnsbl.tornevall.org/scan?ip='

to the DNSBLS hash in vendor/plugins/dnsbl_check/lib/dnsbl_check.rb. Apparently, the http://www.stopforumspam.com/ data is shared with that dnsbl list, and hence can be queried with the same kind of dnsbl lookup.

If that works, we can incorporate that in an update to Instiki. If not, we can look into implementing the API.

 
posted 9 years ago
tanzer 36 posts

Forum: Instiki – Topic: Feature Requests

Background: The Azimuth wiki is getting consistently hit by a spammer promoting “coffre fort.” I’ll do what I can with keyword blocking, but the guy is creative with words.

Main point: The IPs are all in the database at www.stopforumspam.com.

Possible feature request: Option to have this web service called when validating an IP address.

One question is what the performance hit would be – what is the uptime of this service, and what is its response time.

A drawback of doing this is introducing a further dependence through the internet on an external system.

 
posted 9 years ago
distler 123 posts

Forum: Instiki – Topic: Requirements for a server-side Instiki scripting package

You might want to look at this page

% ruby bundle exec rake db:fixtures:export_all
...
% ruby bundle exec rake db:fixtures:import_all

which, mind you, is about exporting a whole Instiki instance (rather than just the pages in a category). But it should give you the general idea.

More generally, it seems that what you want to do is write a few Rake tasks for the sort of administrative functions that you want to perform.

That may be worthwhile, even if you eventually get SSL running properly.

 
posted 9 years ago
distler 123 posts

Forum: Instiki – Topic: How to develop an ABC music plugin for Instiki?

The Markdown+Blahtex engine works similarly: shelling out to Blahtex to produce png images for the equations (which are cached on the server.

So you might look closely at the Blahtex Math Extension in (my version of) Maruku.

 
posted 9 years ago
tanzer 36 posts

Forum: Instiki – Topic: Requirements for a server-side Instiki scripting package

After considering these requirements, and enumerating the commands that are on the Edit Webs page, let’s take a breath, and then move on to discuss the SQL interpretation of these commands in another discussion thread.

Thanks for listening!

 
posted 9 years ago
tanzer 36 posts

edited 9 years ago

Forum: Instiki – Topic: Requirements for a server-side Instiki scripting package

Please if anyone sees any problems with these requirements, or has suggestions for simplifications, enhancements or new use cases – do tell!

 
posted 9 years ago
tanzer 36 posts

Forum: Instiki – Topic: Requirements for a server-side Instiki scripting package

I believe that these specifications cover the story I mentioned above, which is to publish from one installation to another based on category:publish. They also cover more. For instance, multiple people could each have their own staging installation, and with these scripting commands, they could publish page sets to a common target installation.

 
posted 9 years ago
tanzer 36 posts

Forum: Instiki – Topic: Requirements for a server-side Instiki scripting package

c) Import a page set P into I’.

Options: (1) overwrite existing pages, and clobber their histories, (2) append to the existing histories, (3) fail if the pages already exist.

There is a constraint here. If the export contains full histories of pages, it doesn’t make sense to use option (2) to append these histories to an already existing history. So if option 2 is specified, then an error should be generated if the imported data contains full histories and the targets already exist.

 
posted 9 years ago
tanzer 36 posts

Forum: Instiki – Topic: Requirements for a server-side Instiki scripting package

b) Delete the page set for (W,C) from the target installation I’.

 
posted 9 years ago
tanzer 36 posts

edited 9 years ago

Forum: Instiki – Topic: Requirements for a server-side Instiki scripting package

Functions to support Use-Case 2.

a) Export. Given an Instiki instance I, define a page set by PageSet(W,C), where W is a list of web identifiers and C is a set of categories. This consists of all pages in webs W that fall under one of the categories in C.

The export function can be given a list of page-set-specifiers (W1,C1), …, (WN,CN). Allow the wildcard specification * for any of the arguments Wi or Ci.

Allow the export to also specify whether the full history should be exported, or just the most recent versions of the pages.

Output of the export statement may be based arounda collection of SQL insert statements, which can be applied to the database for another Instiki installation I’. (To keep the levels clear, let’s defer talk about the SQL specifics to a followup thread.) But as the following functions suggest, some more meta-data may be needed.

 
posted 9 years ago
tanzer 36 posts

Forum: Instiki – Topic: Requirements for a server-side Instiki scripting package

Functions to support Use-Case 1.

TODO: just read off the functions from the Edit Webs page, and specify them as abstract commands here.

 
posted 9 years ago
tanzer 36 posts

Forum: Instiki – Topic: Requirements for a server-side Instiki scripting package

Use Case 3: Develop new and more powerful methods for manipulating the data that constitutes Instiki webs.

This is open-ended, and ideas can grow here as we go along.

 
posted 9 years ago
tanzer 36 posts

Forum: Instiki – Topic: Requirements for a server-side Instiki scripting package

The story behind use-case 2.

I maintain one Instiki installation that is competely private, password protected, and not referenced from anywhere on the internet. This is the staging area for Instiki webs that am planning to publish. For some things, I’d rather not think aloud in the open, and would prefer to release when I deem them completed.

I will maintain another Instiki installation for the published webs. It’s also a security precaution, to make sure that my staging webs won’t get compromised if the public installation gets hacked, attacked or spammed.

Now in the staging webs, I will build a full image of what the published webs will contain. To accomplish this, I will just add the line “category:publish:” to any page in a staging web that I want to get published.

So: I’ll want a scripting function that exports all of the data from one installation that is tagged with “publish”, and another one that imports the data into an existing Instiki installation.

 
posted 9 years ago
tanzer 36 posts

edited 9 years ago

Forum: Instiki – Topic: Requirements for a server-side Instiki scripting package

Use Case 2: I want to be able to write scripts to export subsets of the data on one Instiki installation, and import them into other Instiki installations.

 
posted 9 years ago
tanzer 36 posts

edited 9 years ago

Forum: Instiki – Topic: Requirements for a server-side Instiki scripting package

Sidenote: The saga behind use case 1.

The concrete thing which is motivating me here is that I am running Instiki on hosted server account, and want to manage it securely using https, but my host – RailsPlayground – which has been fantastic in every regard except for this one is having a bunch of technical problems getting https to work well. First, they don’t yet support SNI (server name identification) in Apache, so a dedicated IP, for 2 bucks per month, is required in order to get SSL to work. But worse than that is that they are claiming that the dedicated IP, along with a wildcard SSL certificate for the whole domain, will not work for the whole domain – that actually I’ll need a 2 dollar per month dedicated IP for each application running on each subdomain. On top of this, they say that for each login, they can only provide one dedicated IP!

I was also willing to use lynx and regular http to localhost, but lynx isn’t supported on this shared host. It’s rather confused and annoying. I’ve pleaded my case with them, and the admins are sympathetic, so they are going to escalate the issue internally.

In all fairness, I should say that overall I’m really happy with their service, which provides 5 shell/cPanel accounts, lots of bandwidth and ok storage. Each account gets 200G per month of bandwidth, and 6G of storage. Stuff is backed up (so they say), and custom php.ini is supported. Leaving https aside, these guys are real experts on managing Rails applications, and general system administration, and their level of responsiveness is impressive. The Azimuth wiki has been running without incident (well, just one brief incident, where the web server had to be restarted because Passenger stopped spawning processes), appears more responsive than it was on the previous VPS (at RailsPlaygrond also) that was running the Azimuth wiki, and does not manifest the “Smoke” error to do with growing cookies.

But even apart from this Saga, I rather like writing scripts to do things, so I’d be pretty happy administering Instiki this way.

 
posted 9 years ago
tanzer 36 posts

edited 9 years ago

Forum: Instiki – Topic: Requirements for a server-side Instiki scripting package

Use Case 1. I want to write scripts to perform the standard administrative operations that are made available from the Edit Web page of Instiki.

 
posted 9 years ago
tanzer 36 posts

Forum: Instiki – Topic: Requirements for a server-side Instiki scripting package

There are three general use cases that motivate this feature for me.

 
posted 9 years ago
tanzer 36 posts

edited 9 years ago

Forum: Instiki – Topic: Requirements for a server-side Instiki scripting package

I would like to develop a server-side scripting packaging for managing Instiki webs, which will be based on direct calls to the database.

If anyone knows of such a thing already, or even the rudiments of it, do tell – because I’d be glad to use something already existing that meets the bill.

In this thread, I will spell out the requirements that I have in mind. I welcome any comments or suggested modifications to these requirements. Then, in a followup thread, we can talk about how these requirements (for commands) will get translated to SQL update and query statements. Then, in another thread, we can talk about implementation approaches (at the moment I’m leaning towards using Python along with its interface to Mysql).

In any case, on with the requirements…

 
posted 9 years ago
tanzer 36 posts

Forum: Instiki – Topic: How to develop an ABC music plugin for Instiki?

Thanks for the good information.

All I’ve worked with is the script abc2ly.py, which comes with Liliypond. It parses ABC pretty well – though not perfectly. It’s fairly simple, in that it generates a Lilypond script. Then the “fat” app Lilypond does the heavy lifting, when given this script as input, it produces a PDF with beautifully typeset music notation.

So a klunky solution would be to have a Maruku ABC extension that just calls this script, then fires off a Lilypond process, and stores the pdf somewhere in the file area. It would be good to employ some sort of caching scheme for the rendered files, so that this bloated process would only be run once for each exact instance of an ABC source file on a given page. To spell out the idea here more specifically, a crude implementation could be achieved by having a directory for each instiki page, which would contain the source text for all ABC snippets on that page, along with their associated rendered PDF files. When a page is reprocessed, it would match the new ABC snippets against the stored ones, and use the rendered PDF files if there was an exact match. Of course there are nicer ways to do this, but what I just said does define the idea.

If the Maruku filter generates PDF, can that be embedded in the Instiki output in a nice way?

This solution is klunky in the sense that it shells out to some heavy processes, and requires them to be installed on the server. On the other hand, there is a coding minimalism to it – music typesetting is a huge challenge, especially to do well – in that the whole buck gets passed to Lilypond.

This could be a default approach in the absence of something nice that would be more integrated with the Wiki.

 
posted 9 years ago
distler 123 posts

Forum: Instiki – Topic: How to develop an ABC music plugin for Instiki?

There is a hierarchy of processing of inputs in Instiki

  • The chunk handler processes various bits of “Wiki Syntax.” It is extensible, by writing new handlers and including them, by name, in lib/wiki_content.rb.
  • A content engine (Markdown, by default, but Textile and RDoc are also available). New content engines can be added in lib/chunks/engines.rb.
  • Within one of the Markdown content engines, itextomml handles equations.

Maruku (our Markdown processor) has an extension mechanism. Math support is implemented as an extension. But so is the div syntax, as are fenced codeblocks and citations.

Presumably, what you want is an ABC extension for Maruku. That would be much easier to create, if there were a Rubygem implementation of ABC notation. But there doesn’t seem to be one.

Is there a good library in Python/Perl/… that might be worth porting to Ruby?

 
posted 9 years ago
tanzer 36 posts

Forum: Instiki – Topic: How to develop an ABC music plugin for Instiki?

Does Instiki have a general interface / API designed for writing plugins?

 
posted 9 years ago
tanzer 36 posts

edited 9 years ago

Forum: Instiki – Topic: How to develop an ABC music plugin for Instiki?

I’m interested in using Instiki for a wiki that includes sheet music. I like the ABC notation, which is a pure ASCII format, with a defined standard, that’s widely used.

For example, see this link for a sample of the format, the output it renders. (It’s also a really nice Balkan song, Jovano Jovanke.) The ABC converters also generally produce MIDI also, but this is of secondary interest to me.

I realize that this request is out of the context of a math-focused wiki, so potentially the only developer for this would be me. Not sure if I’m up for making the leap to Ruby / rails development, but possibly…

As a starting point, I’m interested in discussing here how one might go about implementing this as an extension / plugin to Instiki.

For reference, PmWIki implements an ABC plugin, here.

But I want to stick with – Instiki.

Still it would be interesting to see how they accomplish it.

A long time ago I wrote an ABC plugin for MediaWiki. I piggybacked on a script abc2ly which is distributed with LilyPond (GPL music typesetting program). This script takes an ABC file as input, and generates a PDF file. My extension was simple but heavy-handed – it called the script, cached up the generated PDF file, and then included it on the page. (Many years later they archived this extension, for being unsupported and a security risk.

Another source of ideas could be from the Traditional Tune Archive, which uses Semantic MediaWiki, and has an ABC plugin. That page has a link to an open source javascript component for rendering ABC notation. However, when I clicked on the link for the actual wiki, it took me to a page with some music and text on it, and then…froze completely. Which led me to wonder whether that whole rig is half-baked.

 
posted 9 years ago
tanzer 36 posts

Forum: Instiki – Topic: Is there a server-side file-cache for internal files?

Answer to my question: there is an application caching mechanism in place.

Touching tmp/restart.txt forces it to restart.

 
posted 9 years ago
tanzer 36 posts

Forum: Instiki – Topic: Is there a server-side file-cache for internal files?

Hi I’m using Instiki with Passenger, on a hosted account.

In config/database.yml, I had the incorrect credentials set, let’s say that I had the username coded as abc.

When I go to the Instiki home page, I get the Smoke error. Checking in log/production.log, I see the error message about the wrong credentials abc.

Then I go to fix database.yml, to change the username to xyz.

However, I get the same Smoke error, and the log file still complains about username abc.

Then I go out for a long walk, and when I come back, it fails with a message about xyz.

Do you know of any components (Passenger?) that would be caching up the contents of the database.yml file, and then expiring the cache after some time?