[php-src] Issue #7863: Replace Libxml2 by Gumbo or Lexbor for HTML parsing
| From: | woho | Date: | Fri, 31 Dec 2021 21:14:47 +0000 |
| Subject: | [php-src] Issue #7863: Replace Libxml2 by Gumbo or Lexbor for HTML parsing | ||
| Groups: | php.bugs | ||
| Request: | Send a blank email to php-bugs+get-238691@lists.php.net to get a copy of this message | ||
Issue: https://github.com/php/php-src/issues/7863
Author: woho
### Description
DOMDocument->loadHTML uses Libxml2 and is fundamentally broken. Valid HTML documents exist that
cannot be parsed correctly and there are no simple workaround that work in all cases.
Specifically:
- Closing HTML tag inside script contents, which is not a rare occurrence, break the parser. Fixing
this for all possible cases by pre-processing is not possible without complete tokenization of the
document, because strings like "<script>" can occur in attribute values.
- Encodings are poorly handled. Libxml2 only reliably supports UTF-8 and ISO-8859-1, but other
encodings such as ISO-8859-2 are popular for some languages. Workarounds are very dirty, the easiest
one is to identify the encoding by parsing the document twice and convert the whole document using
mb_convert_encoding to 'HTML-ENTITIES'.
There are two good HTML parsing libraries written in C,
[Gumbo](https://github.com/google/gumbo-parser) and [Lexbor](https://github.com/lexbor/lexbor).
Their performance is similar. [For Gumbo, a PHP extension already
exists](https://github.com/layershifter/gumbo-php), which integrates seamlessly with DOMXPath. It
would be great to have a properly working native solution for loadHTML in PHP, after all it is a web
development language.