Re: Re: [RFC] [Discussion] DOM HTML5 parsing and serialization support
| From: | Niels Dossche | Date: | Fri, 29 Sep 2023 16:38:27 +0000 |
| Subject: | Re: Re: [RFC] [Discussion] DOM HTML5 parsing and serialization support | ||
| References: | 1 2 3 4 5 | Groups: | php.internals |
| Request: | Send a blank email to internals+get-121176@lists.php.net to get a copy of this message | ||
Hi Tim
On 29/09/2023 18:06, Tim Düsterhus wrote:
> Hi
>
> On 9/29/23 17:45, Niels Dossche wrote:
>> Right, we follow the HTML spec in this regard. Roughly speaking we determine the charset in
>> the following order of priorities.
>> If one option fails, it will fall through to the next one.
>> 1. The Content-Type HTTP header from which you loaded the document.
>
> How would the new document classes make use of that? The HTTP header is transmitted out-of-band
> with regard to the actual payload.
>
> Is this referring to passing a
http://` path
> to HTMLDocument::createFromFile()? This would be unusable for everyone who manually downloads the
> document, e.g. using a PSR-18 HTTP Client.
When the stream wrapper contains header information that information is used indeed.
That would unfortunately indeed mean it's unusable when manually passed in.
>
> It might actually be necessary to add an encoding parameter to these functions, but it would
> need to take priority over anything implicit. The current $encoding of the global \DOMDocument has
> the problem that it doesn't take priority/is ignored entirely. Manually converting the document
> to UTF-8 before passing it to \DOMDocument has the problem that the meta tag in the document takes
> priority.
>
> In fact I've run into this issue before for the implementation of a rich embed feature.
> We're downloading the websites using Guzzle and attempt to make sense of them with
> \DOMDocument. However we can't reliably force the encoding given within the
> 'content-type' response header, so in some cases we obtain mojibake.
>
> This encoding parameter would likely need to be ?string $encoding =
> null with everything non-null overwriting implicit detection and null meaning
> implicit detection in the order of priorities you mentioned.
I agree.
I'll add the optional arguments ?string $override_encoding = null to
XML/HTMLDocument::createFromString and XML/HTMLDocument::createFromFile.
I'd call it override_encoding to emphasize it's about overriding the behaviour.
>
>> 2. BOM sniffing in the content. I.e. UTF-8 with BOM and UTF-16 LE/BE prepend the content
>> with byte markers. This is used to detect encoding.
>> 3. Meta tag in the content.
>
> Best regards
> Tim Düsterhus
Kinds regards
Niels