Instiki uses MathJax to render mathematics on browsers that don’t support MathML (either natively, or through a plugin). That’s slow (and, with mixed MathML/SVG content, somewhat buggy) so it’s preferable to have a browser that supports MathML and SVG natively.
In practice, that mean one of the Mozilla family of browsers. Firefox and Seamonkey, in their latest incarnations, both work very well for rendering XHTML+MathML+SVG content. Safari 10.1, and later, has good-enough MathML support. Earlier versions use MathJax.
The STIX Two fonts are what you need to render MathML. Instiki 0.20.0 offers up those fonts as web-fonts, which means that you don’t need to have them installed on your system. Of course, if you do have them installed, Instiki pages will render a wee bit faster as you won’t need to download the fonts over the internet. So you probably want to install the STIX Two fonts.
The MathPlayer plugin provides MathML support for IE < 11. Apparently, it no longer works in IE 11 or later.
Whatever you installed, head over to Jason Blevins’s test site, to give your new MathML-enabled installation a workout.