#35655 [Asn->Csd]: Missing end-of-line token

ID: 35655 Updated by: iliaa@php.net Reported By: tomas_matousek at hotmail dot com -Status: Assigned +Status: Closed Bug Type: Scripting Engine problem Operating System: * PHP Version: 5CVS-2005-12-13 (snap) Assigned To: iliaa New Comment: This bug has been fixed in CVS. Snapshots of the sources are packaged every three hours; this change will be in the next snapshot. You can grab the snapshot at http://snaps.php.net/. Thank you for the report, and for helping us make PHP better. Previous Comments: ------------------------------------------------------------------------ [2005-12-13 13:34:57] tony2001@php.net Ilia, please check it out. The proposed patch looks good for me. ------------------------------------------------------------------------ [2005-12-13 11:31:47] tomas_matousek at hotmail dot com The same result with latest snapshot. The problem is probably here (file zend_language_scanner.l, around line 1750): if (unput_semicolon) { unput(';'); } I would suggest to replace unputting with yyless(label_len); which would preserve both ';' and {NEWLINE}. ------------------------------------------------------------------------ [2005-12-13 10:54:25] tomas_matousek at hotmail dot com Correction: Expected result of highlighting: $code = ' <? $x = <<<EOT some string EOT $y = 2; ?>' Actual result: $code = ' <? $x = <<<EOT some string EOT $y = 2; ?>' ------------------------------------------------------------------------ [2005-12-13 10:52:46] tomas_matousek at hotmail dot com Description: ------------ PHP lexer omits end-of-line characters following T_END_HEREDOC token. This also causes incorrect output of highlight_string() function. Reproduce code: --------------- $code = ' <? $x = <<<EOT some string EOT $y = 2; ?>'; display_tokens(token_get_all($code)); highlight_string($code); function display_tokens($tokens) { foreach ($tokens as $token) { if (is_array($token)) { echo token_name($token[0]),"\n"; echo $token[0]," '",htmlentities(addcslashes($token[1],"\n\r\t")),"'\n"; } else { echo " '",htmlentities($token),"'\n"; } } } Expected result: ---------------- T_INLINE_HTML 311 '\r\n' T_OPEN_TAG 367 '<?' T_WHITESPACE 370 ' \r\n ' T_VARIABLE 309 '$x' T_WHITESPACE 370 ' ' '=' T_WHITESPACE 370 ' ' T_START_HEREDOC 371 '<<<EOT\r\n' T_STRING 307 'some' T_ENCAPSED_AND_WHITESPACE 314 ' ' T_STRING 307 'string' T_ENCAPSED_AND_WHITESPACE 314 ' \r\n' T_END_HEREDOC 372 'EOT' T_WHITESPACE 370 '\r\n ' // '\r\n' is missing in actual result T_VARIABLE 309 '$y' T_WHITESPACE 370 ' ' '=' T_WHITESPACE 370 ' ' T_LNUMBER 305 '2' ';' T_WHITESPACE 370 '\r\n' T_CLOSE_TAG 369 '?>' <? $x = <<<EOT some string EOT $y = 2; ?> Actual result: -------------- T_INLINE_HTML 311 '\r\n' T_OPEN_TAG 367 '<?' T_WHITESPACE 370 ' \r\n ' T_VARIABLE 309 '$x' T_WHITESPACE 370 ' ' '=' T_WHITESPACE 370 ' ' T_START_HEREDOC 371 '<<<EOT\r\n' T_STRING 307 'some' T_ENCAPSED_AND_WHITESPACE 314 ' ' T_STRING 307 'string' T_ENCAPSED_AND_WHITESPACE 314 ' \r\n' T_END_HEREDOC 372 'EOT' // missing '\r\n' T_WHITESPACE 370 ' ' T_VARIABLE 309 '$y' T_WHITESPACE 370 ' ' '=' T_WHITESPACE 370 ' ' T_LNUMBER 305 '2' ';' T_WHITESPACE 370 '\r\n' T_CLOSE_TAG 369 '?>' <? $x = <<<EOT some string EOT $y = 2; ?> ------------------------------------------------------------------------ -- Edit this bug report at http://bugs.php.net/?id=35655&edit=1