Doc #71207 [Asn]: token_get_all does not recognize context-sensitive keywords

Edit report at https://bugs.php.net/bug.php?id=71207&edit=1 ID: 71207 Updated by: tpunt@php.net Reported by: kontakt at kukulich dot cz Summary: token_get_all does not recognize context-sensitive keywords Status: Assigned Type: Documentation Problem Package: Unknown/Other Function PHP Version: 7.0.1 Assigned To: tpunt Block user comment: N Private report: N New Comment: Resolved with commit: https://svn.php.net/viewvc?view=revision&revision=338345 Previous Comments: ------------------------------------------------------------------------ [2015-12-24 10:23:35] tpunt@php.net Automatic comment from SVN on behalf of tpunt Revision: http://svn.php.net/viewvc/?view=revision&amp;revision=338345 Log: Resolve doc bug #71207 ------------------------------------------------------------------------ [2015-12-23 22:38:08] nikic@php.net Switching to doc bug as the new parameter seems to be undocumented. ------------------------------------------------------------------------ [2015-12-23 22:37:29] nikic@php.net If you want these to be recognized, you must pass the TOKEN_PARSE flag as the second argument to token_get_all(). ------------------------------------------------------------------------ [2015-12-23 22:33:21] kontakt at kukulich dot cz Description: ------------ Thanks to https://wiki.php.net/rfc/context_sensitive_lexer it's possible in PHP7 to use a lot of keywords as constants, methods etc. However token_get_all doesn't return the right tokens. Test script: --------------- <?php $tokens = token_get_all('<?php class Test {const PUBLIC=0;}'); foreach ($tokens as $token) { if (is_array($token)) { echo token_name($token[0]) . "\n"; } } Expected result: ---------------- T_OPEN_TAG T_CLASS T_WHITESPACE T_STRING T_WHITESPACE T_CONST T_WHITESPACE T_STRING T_LNUMBER Actual result: -------------- T_OPEN_TAG T_CLASS T_WHITESPACE T_STRING T_WHITESPACE T_CONST T_WHITESPACE T_PUBLIC T_LNUMBER ------------------------------------------------------------------------ -- Edit this bug report at https://bugs.php.net/bug.php?id=71207&edit=1