[php-src] PR #23959: ext/mbstring: Don't emit surrogates when encoding to UTF-8
| From: | nicolas-grekas | Date: | Mon, 28 Sep 2026 08:08:38 +0000 |
| Subject: | [php-src] PR #23959: ext/mbstring: Don't emit surrogates when encoding to UTF-8 | ||
| Groups: | php.git-pulls | ||
| Request: | Send a blank email to git-pulls+get-38873@lists.php.net to get a copy of this message | ||
Pull Request: https://github.com/php/php-src/pull/23959
Author: nicolas-grekas
mb_convert_encoding("\0\0\xD8\0", 'UTF-8', 'UCS-4')
returns "\xED\xA0\x80", which isn't valid UTF-8. Since 4427b2e1ab6 (PHP
8.3), mbstring also flags its UTF-8 output as valid, so mb_check_encoding(),
mb_scrub(), preg_match('//u') and others accept that string
while they reject the same bytes written as a literal. The same happens with UCS-2 and HTML-ENTITIES
input, mb_convert_variables(), mb_decode_numericentity() and
mb_decode_mimeheader().
The UTF-8 encoder now replaces surrogates with the substitute character, as it already does for
codepoints above U+10FFFF. The internal mode used by mb_strpos() and friends keeps
encoding them, so that different surrogates in a UCS-2 string still don't match each other.
Found while working on #23957.