#46957 [NEW]: The tokenizer returns deprecated values

From: fjas at free dot fr Operating system: Win XP PHP version: 5.3.0alpha3 PHP Bug Type: Unknown/Other Function Bug description: The tokenizer returns deprecated values Description: ------------ token_get_all() displays an old set of constants, not the current 5.3 ones. For instance, T_NS_SEPARATOR is not defined. Thus, the symbol \ is not recognized by the tokenizer (UNKNOWN), and namespaced PHP code is not correctly parsed by token_get_all() --- PHP Version 5.3.0alpha4-dev Build date:Dec 27 2008 22:05:04 Reproduce code: --------------- foreach(token_get_all('<? namespace A\B\C;?>') as $c) { if(is_array($c)) { print(token_name($c[0]))."\n"; } } Expected result: ---------------- T_OPEN_TAG T_WHITESPACE T_NAMESPACE T_WHITESPACE T_STRING T_NS_SEPARATOR T_STRING T_NS_SEPARATOR T_STRING T_CLOSE_TAG Actual result: -------------- T_OPEN_TAG T_WHITESPACE T_NAMESPACE T_WHITESPACE T_STRING UNKNOWN T_STRING UNKNOWN T_STRING T_CLOSE_TAG -- Edit bug report at http://bugs.php.net/?id=46957&edit=1 -- Try a CVS snapshot (PHP 5.2): http://bugs.php.net/fix.php?id=46957&r=trysnapshot52 Try a CVS snapshot (PHP 5.3): http://bugs.php.net/fix.php?id=46957&r=trysnapshot53 Try a CVS snapshot (PHP 6.0): http://bugs.php.net/fix.php?id=46957&r=trysnapshot60 Fixed in CVS: http://bugs.php.net/fix.php?id=46957&r=fixedcvs Fixed in CVS and need be documented: http://bugs.php.net/fix.php?id=46957&r=needdocs Fixed in release: http://bugs.php.net/fix.php?id=46957&r=alreadyfixed Need backtrace: http://bugs.php.net/fix.php?id=46957&r=needtrace Need Reproduce Script: http://bugs.php.net/fix.php?id=46957&r=needscript Try newer version: http://bugs.php.net/fix.php?id=46957&r=oldversion Not developer issue: http://bugs.php.net/fix.php?id=46957&r=support Expected behavior: http://bugs.php.net/fix.php?id=46957&r=notwrong Not enough info: http://bugs.php.net/fix.php?id=46957&r=notenoughinfo Submitted twice: http://bugs.php.net/fix.php?id=46957&r=submittedtwice register_globals: http://bugs.php.net/fix.php?id=46957&r=globals PHP 4 support discontinued: http://bugs.php.net/fix.php?id=46957&r=php4 Daylight Savings: http://bugs.php.net/fix.php?id=46957&r=dst IIS Stability: http://bugs.php.net/fix.php?id=46957&r=isapi Install GNU Sed: http://bugs.php.net/fix.php?id=46957&r=gnused Floating point limitations: http://bugs.php.net/fix.php?id=46957&r=float No Zend Extensions: http://bugs.php.net/fix.php?id=46957&r=nozend MySQL Configuration Error: http://bugs.php.net/fix.php?id=46957&r=mysqlcfg