Bug #80462 [Opn->Csd]: Nullsafe operator tokenize with TOKEN_PARSE flag fails

From: Date: Wed, 02 Dec 2020 09:19:54 +0000
Subject: Bug #80462 [Opn->Csd]: Nullsafe operator tokenize with TOKEN_PARSE flag fails
References: 1  Groups: php.bugs 
Request: Send a blank email to php-bugs+get-230786@lists.php.net to get a copy of this message 
Edit report at https://bugs.php.net/bug.php?id=80462&edit=1

 ID:                 80462
 Updated by:         nikic@php.net
 Reported by:        possumfromspace at gmail dot com
 Summary:            Nullsafe operator tokenize with TOKEN_PARSE flag
                     fails
-Status:             Open
+Status:             Closed
 Type:               Bug
 Package:            *General Issues
 Operating System:   ubuntu
 PHP Version:        8.0.0
 Block user comment: N
 Private report:     N

 New Comment:

Automatic comment on behalf of nikita.ppv@gmail.com
Revision: http://git.php.net/?p=php-src.git;a=commit;h=7a61984a2bf42f2632a8c62245a8496a3e2009cd
Log: Fixed bug #80462


Previous Comments:
------------------------------------------------------------------------
[2020-12-02 08:51:50] possumfromspace at gmail dot com

Description:
------------
On:
PHP 8.0.0 (cli) (built: Dec  1 2020 04:01:52) ( NTS )
Copyright (c) The PHP Group
Zend Engine v4.0.0-dev, Copyright (c) Zend Technologies


When tokenizing as following:

```
<?php token_get_all('<?php $foo = $a?->b();', TOKEN_PARSE);
```

the T_NULLSAFE_OBJECT_OPERATOR is missing

https://3v4l.org/t5aAA

Test script:
---------------
```
<?php token_get_all('<?php $foo = $a?->b();', TOKEN_PARSE);
```

results in something like:
```

array(10) {
  [0]=>
  array(3) {
    [0]=>
    int(390)
    [1]=>
    string(6) "<?php "
    [2]=>
    int(1)
  }
  [1]=>
  array(3) {
    [0]=>
    int(315)
    [1]=>
    string(4) "$foo"
    [2]=>
    int(1)
  }
  [2]=>
  array(3) {
    [0]=>
    int(393)
    [1]=>
    string(1) " "
    [2]=>
    int(1)
  }
  [3]=>
  string(1) "="
  [4]=>
  array(3) {
    [0]=>
    int(393)
    [1]=>
    string(1) " "
    [2]=>
    int(1)
  }
  [5]=>
  array(3) {
    [0]=>
    int(315)
    [1]=>
    string(2) "$a"
    [2]=>
    int(1)
  }
  [6]=>
  array(3) {
    [0]=>
    int(311)
    [1]=>
    string(1) "b"
    [2]=>
    int(1)
  }
  [7]=>
  string(1) "("
  [8]=>
  string(1) ")"
  [9]=>
  string(1) ";"
}
```

expected same result as 


```
<?php token_get_all('<?php $foo = $a?->b();');
```


```
array(11) {
  [0]=>
  array(3) {
    [0]=>
    int(390)
    [1]=>
    string(6) "<?php "
    [2]=>
    int(1)
  }
  [1]=>
  array(3) {
    [0]=>
    int(315)
    [1]=>
    string(4) "$foo"
    [2]=>
    int(1)
  }
  [2]=>
  array(3) {
    [0]=>
    int(393)
    [1]=>
    string(1) " "
    [2]=>
    int(1)
  }
  [3]=>
  string(1) "="
  [4]=>
  array(3) {
    [0]=>
    int(393)
    [1]=>
    string(1) " "
    [2]=>
    int(1)
  }
  [5]=>
  array(3) {
    [0]=>
    int(315)
    [1]=>
    string(2) "$a"
    [2]=>
    int(1)
  }
  [6]=>
  array(3) {
    [0]=>
    int(387)
    [1]=>
    string(3) "?->"
    [2]=>
    int(1)
  }
  [7]=>
  array(3) {
    [0]=>
    int(311)
    [1]=>
    string(1) "b"
    [2]=>
    int(1)
  }
  [8]=>
  string(1) "("
  [9]=>
  string(1) ")"
  [10]=>
  string(1) ";"
}
```

note the T_NULLSAFE_OBJECT_OPERATOR token @ index 6 

https://3v4l.org/t5aAA




------------------------------------------------------------------------



--
Edit this bug report at https://bugs.php.net/bug.php?id=80462&edit=1

Thread (2 messages)

« previous php.bugs (#230786) next »