#46957 [Opn->Csd]: The tokenizer returns deprecated values

From: Date: Sat, 27 Dec 2008 23:17:05 +0000
Subject: #46957 [Opn->Csd]: The tokenizer returns deprecated values
References: 1  Groups: php.bugs 
Request: Send a blank email to php-bugs+get-132346@lists.php.net to get a copy of this message 
 ID:               46957
 Updated by:       felipe@php.net
 Reported By:      fjas at free dot fr
-Status:           Open
+Status:           Closed
 Bug Type:         Unknown/Other Function
 Operating System: Win XP
 PHP Version:      5.3.0alpha3
 New Comment:

This bug has been fixed in CVS.

Snapshots of the sources are packaged every three hours; this change
will be in the next snapshot. You can grab the snapshot at
http://snaps.php.net/.
 
Thank you for the report, and for helping us make PHP better.




Previous Comments:
------------------------------------------------------------------------

[2008-12-27 23:00:38] fjas at free dot fr

Description:
------------
token_get_all() displays an old set of constants, not the current 5.3
ones.
For instance, T_NS_SEPARATOR is not defined.
Thus, the symbol \ is not recognized by the tokenizer (UNKNOWN), and
namespaced PHP code is not correctly parsed by token_get_all()

---
PHP Version 5.3.0alpha4-dev
Build date:Dec 27 2008 22:05:04 

Reproduce code:
---------------
foreach(token_get_all('<? namespace A\B\C;?>') as $c)
{
    if(is_array($c))
    {
        print(token_name($c[0]))."\n";
    }
}

Expected result:
----------------
T_OPEN_TAG
T_WHITESPACE
T_NAMESPACE
T_WHITESPACE
T_STRING
T_NS_SEPARATOR
T_STRING
T_NS_SEPARATOR
T_STRING
T_CLOSE_TAG


Actual result:
--------------
T_OPEN_TAG
T_WHITESPACE
T_NAMESPACE
T_WHITESPACE
T_STRING
UNKNOWN
T_STRING
UNKNOWN
T_STRING
T_CLOSE_TAG



------------------------------------------------------------------------


-- 
Edit this bug report at http://bugs.php.net/?id=46957&edit=1

Thread (2 messages)

« previous php.bugs (#132346) next »