Bug #70232 [Ana->Csd]: Incorrect bump-along behavior with \K and empty string match

Edit report at https://bugs.php.net/bug.php?id=70232&edit=1 ID: 70232 Updated by: cmb@php.net Reported by: nhahtdh at gmail dot com Summary: Incorrect bump-along behavior with \K and empty string match -Status: Analyzed +Status: Closed Type: Bug Package: PCRE related PHP Version: 5.6.12 Assigned To: cmb Block user comment: N Private report: N New Comment: Automatic comment on behalf of cmb Revision: http://git.php.net/?p=php-src.git;a=commit;h=b9f23c2152eb635082e43e62a5c395b16f40054e Log: Fix #70232: Incorrect bump-along behavior with \K and empty string match Previous Comments: ------------------------------------------------------------------------ [2015-08-13 11:29:28] cmb@php.net > For old version, I guess the bug can be fixed by using > PCRE_ANCHORED all time, so that we take full control of the > bump-along. We then can rely on (offsets[1] == start_offset) to > check that the match has advanced. I gave that a quick try, and got 35 failing tests. I have not investigated further, as it seems to me that PCRE_NOTEMPTY_ATSTART would not have been introduced, if a general fix would have been so simple. I still prefer to use PCRE_NOTEMPTY_ATSTART instead of PCRE_NOTEMPTY, if it's available, and to simply fall back to PCRE_NOTEMPTY otherwise. This will leave this bug unresolved for PCRE >= 7.2 and < 8.0, but any user who is bitten by this may update their libpcre. After some internal discussion[1] it turned out that lifting the requirements to libpcre >= 8.0 is not an option for now. [1] <http://markmail.org/thread/go2fprpyzq44invj> ------------------------------------------------------------------------ [2015-08-12 08:42:20] nhahtdh at gmail dot com For old version, I guess the bug can be fixed by using PCRE_ANCHORED all time, so that we take full control of the bump-along. We then can rely on (offsets[1] == start_offset) to check that the match has advanced. As for work-around, I'm not sure whether a general work-around exists. It should probably be handled on case-by-case basis. ------------------------------------------------------------------------ [2015-08-11 16:33:12] cmb@php.net Great, thanks! PCRE_NOTEMPTY_ATSTART is exactly what is needed[1]. It's available as of PCRE 8.00 (2009-10-19)[2], so maybe we need to have a workaround for older libpcre, even though this bug would not be fixed for such versions. [1] <https://lists.exim.org/lurker/message/20090911.102109.6e80cce4.pt-BR.html> [2] <http://www.pcre.org/original/changelog.txt> ------------------------------------------------------------------------ [2015-08-11 14:49:12] nhahtdh at gmail dot com Oh, you are right. I forgot that the exec function bumps the match along automatically, so relying on start_offset is a bad idea. Looking at the source code of pcretest, it seems that it uses a different option. Can you check whether the option is usable? http://vcs.pcre.org/pcre/code/branches/pcre16/pcretest.c?view=markup&pathrev=829#l4233 4233 if (use_offsets[0] == use_offsets[1]) 4234 { 4235 if (use_offsets[0] == len) break; 4236 g_notempty = PCRE_NOTEMPTY_ATSTART | PCRE_ANCHORED; 4237 } ------------------------------------------------------------------------ [2015-08-11 12:40:51] cmb@php.net I can confirm the issue, and that this case would be solved by your suggestion (checking for offsets[1] == start_offset). However, that would break some other cases, e.g. preg_replace('/\b/', '*', '(#11/19/2002#)') Expected: string(20) "(#*11*/*19*/*2002*#)" Actual: string(24) "(#**11**/*19**/*2002**#)" ------------------------------------------------------------------------ The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at https://bugs.php.net/bug.php?id=70232 -- Edit this bug report at https://bugs.php.net/bug.php?id=70232&edit=1