#35286 [Asn->Csd]: tokenizer ext drops final comment
| From: | iliaa@php.net | Date: | Mon, 21 Nov 2005 21:32:43 +0000 |
| Subject: | #35286 [Asn->Csd]: tokenizer ext drops final comment | ||
| References: | 1 | Groups: | php.bugs |
| Request: | Send a blank email to php-bugs+get-88643@lists.php.net to get a copy of this message | ||
ID: 35286
Updated by: iliaa@php.net
Reported By: cellog@php.net
-Status: Assigned
+Status: Closed
Bug Type: Scripting Engine problem
Operating System: *
PHP Version: 5CVS-2005-11-19 (cvs)
Assigned To: helly
New Comment:
This bug has been fixed in CVS.
Snapshots of the sources are packaged every three hours; this change
will be in the next snapshot. You can grab the snapshot at
http://snaps.php.net/.
Thank you for the report, and for helping us make PHP better.
Previous Comments:
------------------------------------------------------------------------
[2005-11-19 10:43:34] helly@php.net
Fixed in head
------------------------------------------------------------------------
[2005-11-19 06:53:42] cellog@php.net
better patch - this one does not increment CG(zend_lineno)
Index: zend_language_scanner.l
===================================================================
RCS file: /repository/ZendEngine2/zend_language_scanner.l,v
retrieving revision 1.131.2.3
diff -u -r1.131.2.3 zend_language_scanner.l
--- zend_language_scanner.l 15 Nov 2005 13:29:28 -0000
1.131.2.3
+++ zend_language_scanner.l 19 Nov 2005 05:52:01 -0000
@@ -1465,6 +1465,12 @@
yymore();
}
+<ST_ONE_LINE_COMMENT><<EOF>> {
+ zendlval->value.str.val = yytext; /* no copying - intentional
*/
+ zendlval->value.str.len = yyleng;
+ zendlval->type = IS_STRING;
+ return T_COMMENT;
+}
<ST_ONE_LINE_COMMENT>{NEWLINE} {
zendlval->value.str.val = yytext; /* no copying - intentional
*/
zendlval->value.str.len = yyleng;
------------------------------------------------------------------------
[2005-11-19 04:11:52] cellog@php.net
This patch fixes the issue with no negative side effects
Index: zend_language_scanner.l
===================================================================
RCS file: /repository/ZendEngine2/zend_language_scanner.l,v
retrieving revision 1.131.2.3
diff -u -r1.131.2.3 zend_language_scanner.l
--- zend_language_scanner.l 15 Nov 2005 13:29:28 -0000
1.131.2.3
+++ zend_language_scanner.l 19 Nov 2005 03:11:22 -0000
@@ -1465,6 +1465,7 @@
yymore();
}
+<ST_ONE_LINE_COMMENT><<EOF>> |
<ST_ONE_LINE_COMMENT>{NEWLINE} {
zendlval->value.str.val = yytext; /* no copying - intentional
*/
zendlval->value.str.len = yyleng;
------------------------------------------------------------------------
[2005-11-19 02:10:19] cellog@php.net
the original reproduce script has no closing newline - the newline does
change the behavior
------------------------------------------------------------------------
[2005-11-19 02:04:20] tony2001@php.net
Doesn't drop anything here:
The code:
<?php
$arr = token_get_all("<?php
print 'foo';
# you'll see it
print 'bar';
# but not this one
");
foreach ($arr as $token) {
if (is_array($token)) var_dump($token[1]);
}
?>
The output:
string(6) "<?php "
string(1) "
"
string(5) "print"
string(1) " "
string(5) "'foo'"
string(2) "
"
string(16) "# you'll see it
"
string(5) "print"
string(1) " "
string(5) "'bar'"
string(2) "
"
string(19) "# but not this one
"
------------------------------------------------------------------------
The remainder of the comments for this report are too long. To view
the rest of the comments, please view the bug report online at
http://bugs.php.net/35286
--
Edit this bug report at http://bugs.php.net/?id=35286&edit=1