Bug #66649 [NEW]: token_get_all() does not tag new bracket array syntax as T_ARRAY

From: bishop Operating system: n/a PHP version: 5.4.24 Package: Scripting Engine problem Bug Type: Bug Bug description:token_get_all() does not tag new bracket array syntax as T_ARRAY Description: ------------ token_get_all() identifies the start of an array using array () syntax with T_ARRAY. I expected token_get_all() to do the same for arrays using the new bracket [] syntax. Instead, token_get_all() ignores bracket style array notation altogether. This means that anyone relying on T_ARRAY parsing will not be able to see the new array syntax notation. Test script: --------------- <?php array_map( function ($string) { array_map( function ($token) { if (is_array($token)) { echo token_name($token[0]) . ', '; } }, token_get_all($string) ); echo PHP_EOL; }, array ('<?php $a = array (1);', '<?php $a = [1]') ); Expected result: ---------------- T_OPEN_TAG, T_VARIABLE, T_WHITESPACE, T_WHITESPACE, T_ARRAY, T_WHITESPACE, T_LNUMBER, T_OPEN_TAG, T_VARIABLE, T_WHITESPACE, T_WHITESPACE, T_ARRAY, T_WHITESPACE, T_LNUMBER, Actual result: -------------- T_OPEN_TAG, T_VARIABLE, T_WHITESPACE, T_WHITESPACE, T_ARRAY, T_WHITESPACE, T_LNUMBER, T_OPEN_TAG, T_VARIABLE, T_WHITESPACE, T_WHITESPACE, T_LNUMBER, -- Edit bug report at https://bugs.php.net/bug.php?id=66649&edit=1 -- Try a snapshot (PHP 5.4): https://bugs.php.net/fix.php?id=66649&r=trysnapshot54 Try a snapshot (PHP 5.5): https://bugs.php.net/fix.php?id=66649&r=trysnapshot55 Try a snapshot (trunk): https://bugs.php.net/fix.php?id=66649&r=trysnapshottrunk Fixed in SVN: https://bugs.php.net/fix.php?id=66649&r=fixed Fixed in release: https://bugs.php.net/fix.php?id=66649&r=alreadyfixed Need backtrace: https://bugs.php.net/fix.php?id=66649&r=needtrace Need Reproduce Script: https://bugs.php.net/fix.php?id=66649&r=needscript Try newer version: https://bugs.php.net/fix.php?id=66649&r=oldversion Not developer issue: https://bugs.php.net/fix.php?id=66649&r=support Expected behavior: https://bugs.php.net/fix.php?id=66649&r=notwrong Not enough info: https://bugs.php.net/fix.php?id=66649&r=notenoughinfo Submitted twice: https://bugs.php.net/fix.php?id=66649&r=submittedtwice register_globals: https://bugs.php.net/fix.php?id=66649&r=globals PHP 4 support discontinued: https://bugs.php.net/fix.php?id=66649&r=php4 Daylight Savings: https://bugs.php.net/fix.php?id=66649&r=dst IIS Stability: https://bugs.php.net/fix.php?id=66649&r=isapi Install GNU Sed: https://bugs.php.net/fix.php?id=66649&r=gnused Floating point limitations: https://bugs.php.net/fix.php?id=66649&r=float No Zend Extensions: https://bugs.php.net/fix.php?id=66649&r=nozend MySQL Configuration Error: https://bugs.php.net/fix.php?id=66649&r=mysqlcfg