Bug #66121 [Fbk->Opn]: UTF-8 lookbehinds match bytes instead of characters

Edit report at https://bugs.php.net/bug.php?id=66121&edit=1 ID: 66121 User updated by: danielklein at airpost dot net Reported by: danielklein at airpost dot net Summary: UTF-8 lookbehinds match bytes instead of characters -Status: Feedback +Status: Open Type: Bug Package: PCRE related PHP Version: 5.5.6 Block user comment: N Private report: N New Comment: I originally thought it was a PCRE bug so I submitted one with them (see http://bugs.exim.org/show_bug.cgi?id=1416 for pcretest results). It has been confirmed to not be a PCRE bug so the bug must be in PHP. Previous Comments: ------------------------------------------------------------------------ [2013-11-20 03:07:01] rasmus@php.net Please test this using the pcretest command line tool which is part of the PCRE package. If you can reproduce the issue there too, file the bug with the PCRE project. If you can't, we'll look into it further here. ------------------------------------------------------------------------ [2013-11-20 00:01:31] danielklein at airpost dot net Description: ------------ The test script appears to check every byte position within the UTF-8 encoded character and backtrack until a valid starting byte is found, then check it. If you remove the improperly inserted characters the resulting bytes do encode the original character correctly. Once it has matched the beginning of the string it should then move ahead by a character, not by a byte. Test script: --------------- <?php // Sinhala characters print(preg_replace('/(?<!ක)/u', '*', 'ක') . "\n"); // Works properly print(preg_replace('/(?<!ක)/u', '*', 'ම') . "\n"); // Triggers the bug // English characters print(preg_replace('/(?<!k)/u', '*', 'k') . "\n"); // Works properly print(preg_replace('/(?<!k)/u', '*', 'm') . "\n"); // Works properly ?> Expected result: ---------------- *ක *ම* *k *m* Actual result: -------------- *ක *�*�*�* *k *m* ------------------------------------------------------------------------ -- Edit this bug report at https://bugs.php.net/bug.php?id=66121&edit=1