Bug #78604 [Asn]: token_get_all() does not properly tokenize FOO<?php with short_open_tag=0

Edit report at https://bugs.php.net/bug.php?id=78604&edit=1 ID: 78604 Updated by: requinix@php.net Reported by: tandre@php.net Summary: token_get_all() does not properly tokenize FOO<?php with short_open_tag=0 Status: Assigned Type: Bug Package: Scripting Engine problem PHP Version: 7.4.0RC2 Assigned To: nikic Block user comment: N Private report: N New Comment: https://github.com/php/php-src/commit/0d568b9fd5bab4471219ed249c6c1c378bb46216 Previous Comments: ------------------------------------------------------------------------ [2019-09-28 14:50:10] tandre@php.net Evaluation is also affected » php --no-php-ini -d short_open_tag=0 -a Interactive shell php > eval('?>FOO<?php'); FOO<?php ------------------------------------------------------------------------ [2019-09-28 14:09:51] tandre@php.net Description: ------------ Note that this happened for me when php was configured with short tags, both for the latest php 7.4 and 8.0 (e.g. this is what travis/compile.sh would do) ext/tokenizer/tests/php_tag_only.phpt would fail if php.ini had short_open_tags=0 tested with php 7.4 NTS, ZTS, and php 8.0 NTS (compiled within the last month) Test script: --------------- » php --no-php-ini -d short_open_tag=0 -a Interactive shell php > var_export(token_get_all('FOO<?php')); array ( 0 => array ( 0 => 313, 1 => 'FOO<?php', 2 => 1, ), ) php > echo PHP_VERSION; 7.4.0-dev Expected result: ---------------- Outputs two tokens, one for FOO, and one for <?php Actual result: -------------- Outputs a single token with FOO<?php ------------------------------------------------------------------------ -- Edit this bug report at https://bugs.php.net/bug.php?id=78604&edit=1