#33857 [Com]: token_get_all() inconsistent results?
ID: 33857
Comment by: phpbugs at majiclab dot com
Reported By: sr at brightlight dot ch
Status: Feedback
Bug Type: Scripting Engine problem
Operating System: Mac OS X 10.4.1
PHP Version: 5.0.4
New Comment:
I ran this modified code with the CVS and it seems to work
so far. However, I had problems compiling some required
modules and I don't have enough time to play with those now,
so I had to revert back to 5.0.4. However, since I was
unable to test it extensively, I am not sure if it truly
fixed the bug. It's working on 5.0.4 again now but that's
because I had to restart Apache after reverting to the old
version... :)
So I can't confirm whether it's been fixed yet or not. I
might try to test it out on one of my non-dev boxes that are
just sitting around collecting dust. But just to confirm,
the best debugging I was able to extrapolate (without even
looking at any C code) was this: It seemed that whatever
internals/functions were called by token_get_all() would
have had some kind of boolean to determine whether or not
the current location of the code is within <?php ?> brackets
or not. My best guess is that somehow that variable was
already set to TRUE even though it hasn't encountered the <?
php yet. That would seem to be why the <?php was being
interpreted as a < (less than), a ? (?: operator) and then a
"string" (php being a function or constant name) and then
whitespace rather than just the open tag.
Anyway, I'm sorry I can't provide better testing right now,
but I've got work to do and I can live around this for now.
Previous Comments:
------------------------------------------------------------------------
[2005-08-01 21:45:21] phpbugs at majiclab dot com
I am experiencing similar problems on my Mac OSX 10.4.2 system
(although I don't think it's just Mac related, just a
coincidence probably). Anyway, I am downloading the CVS now
and will try it out in the next couple days. The problem
seems to go away for a bit after restarting Apache, and comes
up randomly, so I might "think" it's gone for a couple days,
and then it might show up later. So I'll try it out for a
couple days and see what happens.
Thanks!
------------------------------------------------------------------------
[2005-07-26 10:27:24] sniper@php.net
Please try using this CVS snapshot:
http://snaps.php.net/php5-latest.tar.gz
For Windows:
http://snaps.php.net/win32/php5-win32-latest.zip
------------------------------------------------------------------------
[2005-07-26 00:04:18] sr at brightlight dot ch
Description:
------------
Obviously the same problem as in Bug #33093, which is set to
bogus (no comment).
When I tokenize the very same code over and over again I
happen to get different tokens. About 9 times out of 10 I
get:
Token'<',
Token '?',
Token T_STRING 'php',
Token T_WHITESPACE '\n',
About 1 time out of 10 Times I get
Token T_OPEN_TAG '<?php\n',
So the first case happens far more often.
By saying the same code btw. I mean that I put the
tokenizing function into a for-loop. So the code doesn't
even get touched.
PHP 5.0.4 (entropy.ch's binary), Apache 2.0.54, Mac OS X
10.4.1 PPC.
The tokenized files encoding (I tried UTF-8 no BOM and ISO-
Latin-1) doesn't seem to play a role. Line endings are
always Unix-style.
------------------------------------------------------------------------
--
Edit this bug report at http://bugs.php.net/?id=33857&edit=1
Thread (4 messages)