Re: Pre-Proposal: Text_Wiki

Hi, all, I've added notes on who's saying what here. [Jones]

I'd like to propose Text_Wiki, a package that [...] allows you to define different source markup rulesets for Wiki text, converts that source to an internal standard, then sends the converted source to an output engine that renders into any number of target formats.

[Hoizey]

I think you should divide this in several parts: - one main package that deals with the "internal standard" (which will be?) - other more specific packages that deal with transformation from another "language" to this standard and/or the other way

[Jones]

That is exactly the plan, although I will say that I do not intend to address XMLish source markups at the start -- I think that is more the domain of XML parsers, not structured text parsers. ...

Finally: can I get some preliminary votes on the package name "Text_Wiki"? Or is this not the proper time?

[Cox]

I'm +1 for this package, allthough I would wait to commit it when the develop gets consistent. If it could parse (or have room for) other wiki implementations would be nice too.

[Jones] It is my plan to allow for multiple source styles, multiple intermediate filters, and multiple output formats. I think I said this earlier, but I'm happy to reiterate, especially now that I'm actually working on it in earnest. Text_Wiki will transform text marked up with a supported Wiki style into any supported output style, most notably HTML, but also (when render classes are written) PDF, RTF, Latex, and so on. Text_Wiki is a "master" class providing an interface to your choice of each of three "support" classes: Text_Wiki_Parse, Text_Wiki_Filter, and Text_Wiki_Render. It applies the rules in each class, in order, to the source text, and returns the parse/filtered/rendered text. The "Parse" class applies a parsing rule set on the source text; you can have a Parse class that parses from Tavi-style structure, or a class that parses from Meatball-style structure, or a class that parses from the original c2.com Wiki rules. In every case, the parser returns an array that matches an internal, abstracted Text_Wiki meta-standard to pass along to Filter and Render. (More on the meta-standard later. Also, there is no plan to support XML-style sources; that's the domain of XML parsers, not Wiki structured-text parsers.) Next, the "Filter" class takes the meta-array and applies filters to selected elements (different Filter classes may apply different instructions). This is the part that checks to see if WikiPageReferences exist, replaces InterWiki references with their proper URLs, colorizes code blocks, applies embedded macros, and so on. This is *not* a page storage mechanism, but it may refer to such if you decide to implement one yourself. (Text_Wiki has no plans to implement a page storage mechanism at this time.) Finally, the "Render" class applies a rendering rule set on the filtered meta-array; you can have a Render class that generates HTML, or PDF, or RTF, or anything else you like. About the internal meta-array standard: The source text will be converted to a series of "raw" elements (which are plain text to be rendered as they are) and "markup" elements (e.g., with names like "strong_start" and "strong_end" to indicate the beginning and end of strongly-emphasized text). The current list of meta-markup names is as follows:

	raw                          raw text
	strong_start|end             strong (bold) inline text
	emphasis_start|end           empasized (italic) inline text
	monospace_start|end          monospaced (teletype) inline text
	heading_start|end            heading title
	newline                      new line or line break
	wiki                         WikiPageName reference
	interwiki                    InterWiki:Page/Name reference
	url                          embedded URL
	code_block                   block of preformatted code
	horizontal_rule              a horizontal rule across the page
	named_anchor                 a named anchor location marker
	bullet_list_start|end        bullet list block
	bullet_item_start|end        item in a bullet list
	numbered_list_start|end      numbered list block
	numbered_item_start|end      item in a numbered list
	definition_list_start|end    definition list block
	definition_term_start|end    term in a definition list
	definition_def_start|end     definition in a definition list

Individual parsers may add to this list, but the additional markups may not be recognized by all Filter and Render classes (unless you write a Filter and Render class just for that Parser, or unless your additional markups are adopted as Text_Wiki meta-standards). Full disclosure: the individual Text_Wiki components are based in whole or in part on the parsing engine within Tavi at http://tavi.sourceforge.net/, originally coded by Scott Moonen. I know the process may sound confusing; for something that looks so simple on the surface, Wiki is amazingly complex to work with. I hope this helps to clarify what I'm working toward. -- pmj