Bug #78978 [Opn]: Tokenizer tokenizes hex numbers as floats on Windows
| From: | requinix@php.net | Date: | Tue, 17 Dec 2019 15:03:07 +0000 |
| Subject: | Bug #78978 [Opn]: Tokenizer tokenizes hex numbers as floats on Windows | ||
| References: | 1 | Groups: | php.bugs |
| Request: | Send a blank email to php-bugs+get-224353@lists.php.net to get a copy of this message | ||
Edit report at https://bugs.php.net/bug.php?id=78978&edit=1
ID: 78978
Updated by: requinix@php.net
Reported by: bugs dot php dot net_nospam at adviesenzo dot nl
Summary: Tokenizer tokenizes hex numbers as floats on Windows
Status: Open
Type: Bug
Package: Scripting Engine problem
Operating System: Windows 7
PHP Version: 7.4.0
Block user comment: N
Private report: N
New Comment:
Are you using the x86 or x64 build?
Previous Comments:
------------------------------------------------------------------------
[2019-12-17 11:17:09] bugs dot php dot net_nospam at adviesenzo dot nl
Description:
------------
Noticed this while working on something for PHP_CodeSniffer where I wasn't getting the same
results for unit tests on Travis versus my local Windows machine.
It appears that hexidecimal integers using the new PHP 7.4 numeric literal separator are tokenized
differently on Linux versus Windows.
They should be tokenized as
T_LNUMBER and are on Linux. See: https://3v4l.org/vlH5j
However on Windows, I'm seeing them get tokenized as T_DNUMBER.
Test script:
---------------
$tokens = token_get_all(
'<?php
$foo = 0xCAFE_F00D;
$foo = 0x42_72_6F_77_6E;'
);
foreach ($tokens as $token) {
if (is_array($token)) {
echo "Line {$token[2]}: ", token_name($token[0]), "
('{$token[1]}')", PHP_EOL;
}
}
Expected result:
----------------
Line 1: T_OPEN_TAG ('<?php
')
Line 2: T_VARIABLE ('$foo')
Line 2: T_WHITESPACE (' ')
Line 2: T_WHITESPACE (' ')
Line 2: T_LNUMBER ('0xCAFE_F00D')
Line 2: T_WHITESPACE (' ')
Line 3: T_VARIABLE ('$foo')
Line 3: T_WHITESPACE (' ')
Line 3: T_WHITESPACE (' ')
Line 3: T_LNUMBER ('0x42_72_6F_77_6E')
Actual result:
--------------
Line 1: T_OPEN_TAG ('<?php
')
Line 2: T_VARIABLE ('$foo')
Line 2: T_WHITESPACE (' ')
Line 2: T_WHITESPACE (' ')
Line 2: T_DNUMBER ('0xCAFE_F00D')
Line 2: T_WHITESPACE ('
')
Line 3: T_VARIABLE ('$foo')
Line 3: T_WHITESPACE (' ')
Line 3: T_WHITESPACE (' ')
Line 3: T_DNUMBER ('0x42_72_6F_77_6E')
------------------------------------------------------------------------
--
Edit this bug report at https://bugs.php.net/bug.php?id=78978&edit=1