[php-src] PR #23200: Skip doc-comment copying and presize token arrays when tokenizing
| From: | staabm | Date: | Mon, 10 Aug 2026 15:04:32 +0000 |
| Subject: | [php-src] PR #23200: Skip doc-comment copying and presize token arrays when tokenizing | ||
| Groups: | php.git-pulls | ||
| Request: | Send a blank email to git-pulls+get-37732@lists.php.net to get a copy of this message | ||
Pull Request: https://github.com/php/php-src/pull/23200
Author: staabm
disclaimer: this change was generated by claude opus. I have little experience with php-src
development
----
The scanner copied every doc comment into CG(doc_comment) even in non-parser mode
(token_get_all/PhpToken::tokenize/highlighting), where nothing consumes it and it is immediately
discarded; only do it when compiling. Also presize the tokenizer result array (one token per ~5
source bytes) and the dedup hash instead of growing them by doubling.