FnHTML

What it is

FnHTML is an HTML5 parsing library built on FnXML. It follows the parsing algorithm in the WHATWG HTML standard, including the error recovery real pages depend on, and emits a stream of well-formed XHTML events. That means the whole XML toolchain (DOM building, SAX, StAX, plain stream processing) works on HTML5 documents without a separate HTML tree API.

In the Fn* document pipeline it is the HTML front end: its output is what FnCSS styles and FnLayout lays out.

Example

# Parse HTML5 to a well-formed XHTML event stream
FnHTML.parse("<p>Hello<p>World")
|> Enum.to_list()

# Build a DOM from HTML5 with FnXML
FnHTML.parse("<div class=container>Content</div>")
|> FnXML.API.DOM.build()

In the first example the second <p> implicitly closes the first, as the standard requires, so the events come out balanced.

How it works

Parsing runs in three stages.

1. Tokenizer. An implementation of the WHATWG tokenization states, covering script data (including double-escaped script), RCDATA elements such as title and textarea, raw-text elements such as script and style, the standard's comment edge cases, DOCTYPE parsing, unquoted and boolean attributes, and end-tag error recovery. Character references are resolved in text and attribute values but not inside raw-text elements. Fragment parsing takes a context element, which sets the tokenizer's starting state.
2. Tree construction. The standard's tree-building rules run over an in-memory tree rather than directly on the event stream. Some of those rules, notably the adoption agency algorithm for misnested formatting elements such as <b><i></b></i>, restructure parts of the document already seen, so events are emitted only once the structure is final. Foreign content (SVG and MathML) is tracked by namespace.
3. XHTML conversion. Tag and attribute names are lowercased, void elements get end tags, implied end tags and the optional html, head and body elements are inserted, and U+0000 is replaced with U+FFFD. The result is valid input for any FnXML consumer.

The events use the same shape as FnXML's, each with its source line and offset, and parse errors are reported as events rather than exceptions.

Testing

Conformance is measured against the html5lib-tests tree-construction suite through a shared conformance runner used across the Fn* libraries. A benchmark suite compares FnHTML's tokenizer, tree building and full pipeline with mochiweb (via Floki) and with html5ever as a spec-conformant reference, recording throughput, reductions and memory per fixture.

Status

FnHTML is experimental and unpublished. It is at version 0.2, its README is still a placeholder, and there are known gaps. For example, the named character reference table is not yet included, so named entities such as   are not decoded. The streaming entry point currently collects its whole input before tokenizing; true incremental streaming is planned. Expect the API to change.