Bug #66121 [NEW]: UTF-8 lookbehinds match bytes instead of characters

From:	danielklein at airpost dot net	Date:	Wed, 20 Nov 2013 00:01:31 +0000
Subject:	Bug #66121 [NEW]: UTF-8 lookbehinds match bytes instead of characters
Groups:	php.bugs
Request:	Send a blank email to php-bugs+get-182845@lists.php.net to get a copy of this message

From:             danielklein at airpost dot net
Operating system: 
PHP version:      5.5.6
Package:          PCRE related
Bug Type:         Bug
Bug description:UTF-8 lookbehinds match bytes instead of characters

Description:
------------
The test script appears to check every byte position within the UTF-8
encoded character and backtrack until a valid starting byte is found,
then check it. If you remove the improperly inserted characters the
resulting bytes do encode the original character correctly.

Once it has matched the beginning of the string it should then move
ahead by a character, not by a byte.

Test script:
---------------
<?php
// Sinhala characters
print(preg_replace('/(?<!ක)/u', '*', 'ක') .
"\n"); // Works
properly
print(preg_replace('/(?<!ක)/u', '*', 'ම') .
"\n"); // Triggers the
bug
// English characters
print(preg_replace('/(?<!k)/u', '*', 'k') . "\n"); //
Works properly
print(preg_replace('/(?<!k)/u', '*', 'm') . "\n"); //
Works properly
?>


Expected result:
----------------
*ක
*ම*
*k
*m*

Actual result:
--------------
*ක
*�*�*�*
*k
*m*

-- 
Edit bug report at https://bugs.php.net/bug.php?id=66121&edit=1
-- 
Try a snapshot (PHP 5.4):   https://bugs.php.net/fix.php?id=66121&r=trysnapshot54
Try a snapshot (PHP 5.5):   https://bugs.php.net/fix.php?id=66121&r=trysnapshot55
Try a snapshot (trunk):     https://bugs.php.net/fix.php?id=66121&r=trysnapshottrunk
Fixed in SVN:               https://bugs.php.net/fix.php?id=66121&r=fixed
Fixed in release:           https://bugs.php.net/fix.php?id=66121&r=alreadyfixed
Need backtrace:             https://bugs.php.net/fix.php?id=66121&r=needtrace
Need Reproduce Script:      https://bugs.php.net/fix.php?id=66121&r=needscript
Try newer version:          https://bugs.php.net/fix.php?id=66121&r=oldversion
Not developer issue:        https://bugs.php.net/fix.php?id=66121&r=support
Expected behavior:          https://bugs.php.net/fix.php?id=66121&r=notwrong
Not enough info:            https://bugs.php.net/fix.php?id=66121&r=notenoughinfo
Submitted twice:            https://bugs.php.net/fix.php?id=66121&r=submittedtwice
register_globals:           https://bugs.php.net/fix.php?id=66121&r=globals
PHP 4 support discontinued: https://bugs.php.net/fix.php?id=66121&r=php4
Daylight Savings:           https://bugs.php.net/fix.php?id=66121&r=dst
IIS Stability:              https://bugs.php.net/fix.php?id=66121&r=isapi
Install GNU Sed:            https://bugs.php.net/fix.php?id=66121&r=gnused
Floating point limitations: https://bugs.php.net/fix.php?id=66121&r=float
No Zend Extensions:         https://bugs.php.net/fix.php?id=66121&r=nozend
MySQL Configuration Error:  https://bugs.php.net/fix.php?id=66121&r=mysqlcfg



   

Thread (11 messages)

• danielklein at airpost dot netWed, 20 Nov 2013 00:01:31 +0000
  • rasmus@php.netWed, 20 Nov 2013 03:07:02 +0000Bug #66121 [Opn->Fbk]: UTF-8 lookbehinds match bytes instead of characters
  • danielklein at airpost dot netWed, 20 Nov 2013 23:31:31 +0000Bug #66121 [Fbk->Opn]: UTF-8 lookbehinds match bytes instead of characters
  • danielklein at airpost dot netFri, 22 Nov 2013 04:14:54 +0000Bug #66121 [Opn]: UTF-8 lookbehinds match bytes instead of characters
  • nhahtdh at gmail dot comMon, 15 Dec 2014 11:19:50 +0000Bug #66121 [Com]: UTF-8 lookbehinds match bytes instead of characters
  • danielklein at airpost dot netTue, 16 Dec 2014 03:28:54 +0000Bug #66121 [Opn]: UTF-8 lookbehinds match bytes instead of characters
  • danielklein at airpost dot netTue, 16 Dec 2014 03:55:37 +0000Bug #66121 [Opn]: UTF-8 lookbehinds match bytes instead of characters
  • nhahtdh at gmail dot comTue, 16 Dec 2014 11:01:26 +0000Bug #66121 [Com]: UTF-8 lookbehinds match bytes instead of characters
  • cmbecker69 at gmx dot deThu, 08 Jan 2015 22:24:51 +0000Bug #66121 [Com]: UTF-8 lookbehinds match bytes instead of characters
  • danielklein at airpost dot netTue, 13 Jan 2015 08:35:31 +0000Bug #66121 [Opn]: UTF-8 lookbehinds match bytes instead of characters
  • cmb@php.netFri, 05 Jun 2015 12:15:02 +0000Bug #66121 [Opn->Dup]: UTF-8 lookbehinds match bytes instead of characters