Wikipedia:Wikipedia Signpost/2008-01-21/Parser changes

On 24 January, a new preprocessor will be introduced that substantially changes the code that is used to parse many sorts of wikimarkup, in particular templates. This will fix a large number of bugs, and change the meaning of certain syntax. The message announcing this change is available here; a longer description is available at m:Migration to the new preprocessor, including a description of which of the changes are and are not negotiable.

The preprocessor is part of the parser that handles certain constructs that are handled before the other constructs on the page are parsed, including section edit links, transclusions, and comments. At present, all the pages run on the old preprocessor by default; but as "[a]ll known bugs in the new preprocessor have been fixed" it is now available as an alternative for testing purposes. It is known that some templates will cause problems with the new preprocessor (in particular, {{self}} has been singled out as a template known to have problems); therefore, the community's help has been requested to find uses of markup that differs between the two preprocessors and to fix it to be more 'correct'.

There are several categories of changes made by the new preprocessor; most of them fix bugs in one way or another, but some are due to differences between the way the two preprocessors operate:

• There have been many changes to the way that section edit links work, so that they work in a more logical manner. The basic rule is that with the new preprocessor, only headers formed using the usual ==syntax== will create a section edit link, or start/end a section, and even then only if they are the only markup on the line (so comments should no longer be placed on the same line as a section header). (<h2>-style headers will still appear in the TOC, but will not count as a section break for section editing purposes.) Sections transcluded from templates will still be editable if the ==syntax== is used in the template (this provides a way to choose whether or not transcluded sections are editable), but the section edit links will contain the prefix T- rather than just being numbers.
• The way that <!--comments--> are handled has also changed. It's now possible to comment out <noinclude> or <includeonly>, and comments will only swallow up line breaks before and after them (meaning that newline-comment-newline becomes one newline not two) if there is only one comment on the line.
• There have been several changes to the way that template parameters are handled, most notably to fix bug 5678. This bug has affected many templates in the past, forcing workarounds to be used (for instance, this happened in the code in the uw- templates for removing excess code after substing):
  • The markup for specifying a template parameter is, as it always was, parameter=value. However, this markup cannot now be produced as a template output and reparsed as template input.
  • The way that errors are handled in parser functions such as #expr: has remained the same, but error codes are no longer reliably caught by #switch:. A new parser function #iferror: has been added to detect if an error has been produced.
  • The use of {{!}} has changed to some extent. It's now no longer possible to use it as a parameter separator, although its other uses stay correct.
  • Some templates relied on bug 5678 to work; as the bug has been fixed, those templates will no longer work.
  • Certain constructs which were dubious originally (such as putting a parser function inside an html tag) now act in a different manner, normally due to expanding the parser function and tag in a different order.
  • Infinite loops are now detected in a different way, so looping code (a template that transcludes itself) may act differently due to the limits being hit.

One notable advantage to the new preprocessor is that branches of a parser function that are not used (such as the b in {{#if:1|a|b}}) are no longer expanded, resulting in lower pre-expand include values and reducing the length of time the server needs to parse such constructs.

Users are encouraged to test the new preprocessor before it goes live; there are two ways to do this. First, appending the option ?timtest=newpp (or &timtest=newpp if using /w/index.php rather than /wiki) to a page's URL will cause it to be rendered with the new rather than the old preprocessor. It is also possible to use Special:ParserDiffTest to check the difference between using the old and the new preprocessor on a page, using either subpage syntax or by entering some wikitext directly. Such testing will help to ensure that the change to the new preprocessor does not break large parts of the wiki when it finally goes live.