Edit report at https://bugs.php.net/bug.php?id=81390&edit=1
ID: 81390
Updated by: alexdowad@php.net
Reported by: alec at alec dot pl
Summary: mb_detect_encoding() regression
Status: Assigned
Type: Bug
Package: mbstring related
PHP Version: 8.1.0beta5
Assigned To: alexdowad
Block user comment: N
Private report: N
New Comment:
Alec, you are nothing short of a fabulous tester. Well done.
I am going to investigate your latest finding... just not right now. At the moment, I am cooking
something up which will hopefully make almost every mbstring function several times faster. Soon to
be served up, hot and fresh, in a upcoming PHP release. Bon appetit!
Previous Comments:
------------------------------------------------------------------------
[2021-11-04 17:35:39] alec at alec dot pl
After testing rc5 I have one last case.
https://3v4l.org/9o7ft
There is a reasonably short input ('Iksi' . chr(241) . 'ski') recognized as
SJIS-2004, while I'd expect ISO-8859-2 or ISO-8859-1. It is also interesting that without the
second argument the result would be UTF-8.
I understand that such a short input may be confusing for the detector, but maybe there's still
some bug.
------------------------------------------------------------------------
[2021-11-01 19:40:34] patrickallaert@php.net
This is part of PHP 8.1.0RC5.
------------------------------------------------------------------------
[2021-11-01 19:35:11] alexdowad@php.net
Closing this for now, as I believe the enhancements which have been merged in to mainline will fully
address Alec's concerns. (Whenever a new public release comes out which he can install and try,
that is.)
Alec, please re-open this if you still discover any problems with the detection accuracy of
mb_detect_encoding() in the next rc release. On the other hand, if it works well for you, please
drop a line as well.
------------------------------------------------------------------------
[2021-10-17 18:48:44] alexdowad@php.net
https://github.com/php/php-src/pull/7587
------------------------------------------------------------------------
[2021-10-17 17:20:44] alexdowad@php.net
Hi Alec,
The test case which you provided in the original bug report does appear to return good results now.
Looks like the bug fix had a good effect:
https://3v4l.org/UXtBd
Thank you for providing more test cases in which the latest implementation of mb_detect_encoding
does not return accurate results. Those will be added to the test suite for the PHP interpreter.
I have been working on another patch for mbstring which significantly enhances the accuracy of
legacy encoding detection. Several of the test cases you have added in this thread already return
the correct result with that new patch.
Once the next patch is merged, we would appreciate your further assistance with testing. If you are
able to build PHP from source, it would be even more helpful if you can test even before the next
rc. However, I understand that building a C program may seem daunting for some who have never done
that before, and it is work which you didn't sign up for.
------------------------------------------------------------------------
The remainder of the comments for this report are too long. To view
the rest of the comments, please view the bug report online at
https://bugs.php.net/bug.php?id=81390
--
Edit this bug report at https://bugs.php.net/bug.php?id=81390&edit=1