Bug #78978 [Opn]: Tokenizer tokenizes hex numbers as floats on Windows

Edit report at https://bugs.php.net/bug.php?id=78978&edit=1 ID: 78978 Updated by: cmb@php.net Reported by: bugs dot php dot net_nospam at adviesenzo dot nl Summary: Tokenizer tokenizes hex numbers as floats on Windows Status: Open Type: Bug Package: Scripting Engine problem Operating System: Windows 7 PHP Version: 7.4.0 Block user comment: N Private report: N New Comment: Thank you for taking the time to write to us, but this is not a bug. Please double-check the documentation available at http://www.php.net/manual/ and the instructions on how to report a bug at http://bugs.php.net/how-to-report.php This is not related to numeric literal seperators (you get the same result without them), but rather to integers overflowing to float (you can var_dump() these values to confirm). Previous Comments: ------------------------------------------------------------------------ [2019-12-17 15:03:07] requinix@php.net Are you using the x86 or x64 build? ------------------------------------------------------------------------ [2019-12-17 11:17:09] bugs dot php dot net_nospam at adviesenzo dot nl Description: ------------ Noticed this while working on something for PHP_CodeSniffer where I wasn't getting the same results for unit tests on Travis versus my local Windows machine. It appears that hexidecimal integers using the new PHP 7.4 numeric literal separator are tokenized differently on Linux versus Windows. They should be tokenized as T_LNUMBER and are on Linux. See: https://3v4l.org/vlH5j However on Windows, I'm seeing them get tokenized as T_DNUMBER. Test script: --------------- $tokens = token_get_all( '<?php $foo = 0xCAFE_F00D; $foo = 0x42_72_6F_77_6E;' ); foreach ($tokens as $token) { if (is_array($token)) { echo "Line {$token[2]}: ", token_name($token[0]), " ('{$token[1]}')", PHP_EOL; } } Expected result: ---------------- Line 1: T_OPEN_TAG ('<?php ') Line 2: T_VARIABLE ('$foo') Line 2: T_WHITESPACE (' ') Line 2: T_WHITESPACE (' ') Line 2: T_LNUMBER ('0xCAFE_F00D') Line 2: T_WHITESPACE (' ') Line 3: T_VARIABLE ('$foo') Line 3: T_WHITESPACE (' ') Line 3: T_WHITESPACE (' ') Line 3: T_LNUMBER ('0x42_72_6F_77_6E') Actual result: -------------- Line 1: T_OPEN_TAG ('<?php ') Line 2: T_VARIABLE ('$foo') Line 2: T_WHITESPACE (' ') Line 2: T_WHITESPACE (' ') Line 2: T_DNUMBER ('0xCAFE_F00D') Line 2: T_WHITESPACE (' ') Line 3: T_VARIABLE ('$foo') Line 3: T_WHITESPACE (' ') Line 3: T_WHITESPACE (' ') Line 3: T_DNUMBER ('0x42_72_6F_77_6E') ------------------------------------------------------------------------ -- Edit this bug report at https://bugs.php.net/bug.php?id=78978&edit=1