Bug #72933 [NEW]: mb_detect_encoding analyzing only the first byte of a string

From: paul dot crovella at gmail dot com Operating system: PHP version: Irrelevant Package: mbstring related Bug Type: Bug Bug description:mb_detect_encoding analyzing only the first byte of a string Description: ------------ In non-strict mode it appears only the first byte of a string is being checked by mb_detect_encoding in some circumstances. For example, the byte 0xf8 is not allowed anywhere in UTF-8. When placed at the start of the string mb_detect_encoding() properly returns false for it regardless of which mode is used. However if any valid UTF-8 byte occurs at the beginning of the string mb_detect_encoding() in non-strict mode will declare it as UTF-8. This is also evident with a string like "\xe1\xe9\xf3\xfa", which is the ISO-8859-1 encoded version of "áéóú". The first byte, 0xe1, is allowed in UTF-8 as the first of a multi-byte character - however the string as a whole is invalid and may not occur. The suspect code is at: https://github.com/php/php-src/blob/c72282a13b12b7e572469eba7a7ce593d900a8a2/ext/mbstring/libmbfl/mbfl/mbfilter.c#L746-L761 The problem exists in all current versions of PHP: https://3v4l.org/b9b6q Test script: --------------- // This returns as expected. $str = "\xf8foo"; var_dump( mb_detect_encoding($str, 'UTF-8'), // bool(false) mb_detect_encoding($str, 'UTF-8', true) // bool(false) ); // This does not. $str = "foo\xf8"; var_dump( mb_detect_encoding($str, 'UTF-8'), // string(5) "UTF-8" mb_detect_encoding($str, 'UTF-8', true) // bool(false) ); // Nor does this. $str = "\xe1\xe9\xf3\xfa"; var_dump( mb_detect_encoding($str, 'UTF-8'), // string(5) "UTF-8" mb_detect_encoding($str, 'UTF-8', true) // bool(false) ); Expected result: ---------------- bool(false) bool(false) bool(false) bool(false) bool(false) bool(false) Actual result: -------------- bool(false) bool(false) string(5) "UTF-8" bool(false) string(5) "UTF-8" bool(false) -- Edit bug report at https://bugs.php.net/bug.php?id=72933&edit=1 -- Try a snapshot (PHP 5.4): https://bugs.php.net/fix.php?id=72933&r=trysnapshot54 Try a snapshot (PHP 5.5): https://bugs.php.net/fix.php?id=72933&r=trysnapshot55 Try a snapshot (trunk): https://bugs.php.net/fix.php?id=72933&r=trysnapshottrunk Fixed in SVN: https://bugs.php.net/fix.php?id=72933&r=fixed Fixed in release: https://bugs.php.net/fix.php?id=72933&r=alreadyfixed Need backtrace: https://bugs.php.net/fix.php?id=72933&r=needtrace Need Reproduce Script: https://bugs.php.net/fix.php?id=72933&r=needscript Try newer version: https://bugs.php.net/fix.php?id=72933&r=oldversion Not developer issue: https://bugs.php.net/fix.php?id=72933&r=support Expected behavior: https://bugs.php.net/fix.php?id=72933&r=notwrong Not enough info: https://bugs.php.net/fix.php?id=72933&r=notenoughinfo Submitted twice: https://bugs.php.net/fix.php?id=72933&r=submittedtwice register_globals: https://bugs.php.net/fix.php?id=72933&r=globals PHP 4 support discontinued: https://bugs.php.net/fix.php?id=72933&r=php4 Daylight Savings: https://bugs.php.net/fix.php?id=72933&r=dst IIS Stability: https://bugs.php.net/fix.php?id=72933&r=isapi Install GNU Sed: https://bugs.php.net/fix.php?id=72933&r=gnused Floating point limitations: https://bugs.php.net/fix.php?id=72933&r=float No Zend Extensions: https://bugs.php.net/fix.php?id=72933&r=nozend MySQL Configuration Error: https://bugs.php.net/fix.php?id=72933&r=mysqlcfg