Bug #61780 [Opn->Ana]: Inconsistent PCRE captures in match results

Edit report at https://bugs.php.net/bug.php?id=61780&edit=1 ID: 61780 Updated by: cmb@php.net Reported by: danielklein at airpost dot net Summary: Inconsistent PCRE captures in match results -Status: Open +Status: Analyzed Type: Bug Package: PCRE related -Operating System: +Operating System: * -PHP Version: 5.4.0 +PHP Version: 5.6.9 -Assigned To: +Assigned To: cmb Block user comment: N Private report: N New Comment: Verified: <http://3v4l.org/Qk9Pm>. From the PCRE manuals[1]: > When any of these functions encounter a substring that is unset, > which can happen when capturing subpattern number n+1 matches > some part of the subject, but subpattern n has not been used at > all, they return an empty string. This can be distinguished from > a genuine zero-length substring by inspecting the appropriate > offset in ovector, which is negative for unset substrings. This disambigution, however, is not done by PHP's implementation. Instead the offsets (there are two for each subpattern: start and end) are always subtracted, what yields 0 for unset offsets, and so a string of length 0 (i.e. an empty string) is appended to the resulting array. Fixing the behavior to use NULL instead of an empty string doesn't seem to be hard, but the change would constitute a BC break (existing code actually might rely on a empty string). It might be okay, however, to change the behavior for PHP 7.0.0. [1] <http://www.pcre.org/original/doc/html/pcreapi.html#SEC18> Previous Comments: ------------------------------------------------------------------------ [2015-05-23 08:42:51] danielklein at airpost dot net Output from pcretest: re> /(4)?(2)?\d/g data> 123456 0: 1 0: 23 1: <unset> 2: 2 0: 45 1: 4 0: 6 data> Equivalent PHP code: <?php preg_match_all('/(4)?(2)?\d/', '123456', $matches, PREG_SET_ORDER); var_export($matches); /* Outputs: array ( 0 => array ( 0 => '1', ), 1 => array ( 0 => '23', 1 => '', // This key should not exist as it is unset, not blank in pcretest 2 => '2', ), 2 => array ( 0 => '45', 1 => '4', ), 3 => array ( 0 => '6', ), ) */ ?> pcretest for original report: re> /(?:(?<b>b)|(?<c>c)|(?<d>d))(?<e>e)?/ data> cdec 0: c 1: <unset> 2: c data> re> /(?:(?<b>b)|(?<c>c)|(?<d>d))(?<e>e)?/g data> cdec 0: c 1: <unset> 2: c 0: de 1: <unset> 2: <unset> 3: d 4: e 0: c 1: <unset> 2: c data> 2: c data Showing the difference between a blank capture and no capture (unset): re> /(a)?(b?)(c)?/ data> a 0: a 1: a 2: data> b 0: b 1: <unset> 2: b data> c 0: c 1: <unset> 2: 3: c data> 2: 3: c data ------------------------------------------------------------------------ [2013-11-04 00:44:54] danielklein at airpost dot net @michael: Your code has a bug in it. There is an extra slash "/" in your pattern. The code works as expected if you remove it. @pajoye: This bug is still happening. It could be a bug in PCRE itself but I don't know how PHP gets its results from it. By the way, I can't set this bug back to Re-Opened. It says: "ERROR: You aren't allowed to change a bug to that state." ------------------------------------------------------------------------ [2013-10-15 11:54:33] php-bugs at lists dot php dot net No feedback was provided. The bug is being suspended because we assume that you are no longer experiencing the problem. If this is not the case and you are able to provide the information that was requested earlier, please do so and change the status of the bug back to "Re-Opened". Thank you. ------------------------------------------------------------------------ [2013-02-11 09:00:19] pajoye@php.net Be sure that every testing environment uses the exact same version of PCRE. PHP only uses it and does not affect by any mean the parsing. ------------------------------------------------------------------------ [2013-02-11 08:56:33] michael at mbaas dot de Here is a reproduceable example (PHP 5.3.20 and 5.3.21) where named captures do not return matches at all! I've tested this pattern against the PCRE- Implementation in another language and it worked... <?php $QQ=chr(92) . chr(34); $delimeters = "{}"; $del0 = preg_quote($delimeters{0}); $del1 = preg_quote($delimeters{1}); $tag="language"; $string="fdfdfdfdf{language=1}testhgg"; $preg = "~" . $del0 . $tag . "\s*=\s*(?P<" . "quote>[" . $QQ . "\']*)(? P<att>.*?)(?P=quote)\s*/" . $del1 . "~"; $match=array(); preg_match($preg,$string,$match); echo "<br>string = " . htmlspecialchars($string) . "<br>preg=" . htmlspecialchars($preg) . "<br>match:<pre>";var_dump($match);echo"</pre>"; ?> ------------------------------------------------------------------------ The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at https://bugs.php.net/bug.php?id=61780 -- Edit this bug report at https://bugs.php.net/bug.php?id=61780&edit=1