Re: Request for opinions: bug vs f eature - change intokenization of yield from

On 21 July 2024 19:33:11 BST, Juliette Reinders Folmer <php-internals_nospam@adviesenzo.nl> wrote: >The crux - to me - is that it is an undocumented breaking change, which by definition is a bug. There are two parts of this which are bugs, in my opinion: - That it wasn't documented, e.g. with a line in UPGRADING listing the affected tokens. - That the tokenisation consumes the comment as part of the token, rather than just performing a lookahead. One is easily fixed; the other is more subtle, but maybe fixable. >As I've said before, I'm not against changing the tokenization, what I'm speaking >up about is that it was done in an inconsistent, semi-random and undocumented way. As others have said, there is nothing unusual in the process that was followed here. A minor change was proposed via Pull Request, discussed with multiple core contributors, and wasn't deemed significant enough for a wider discussion or RFC. The documentation probably *should* have been caught during that review, because it's a common checklist item. The behaviour of the token stream could have been, but we got unlucky and nobody thought of it. There's no guarantee that a different process would have done better - there have been changes which went through a whole RFC process, then a year later someone points out a flaw that could have been avoided; that's life. Now that we have spotted it, we need to decide what to do. Regards, Rowan Tommins [IMSoP]