Re: Revert Tokenizer behavior for 5.4

Hi Hannes, thanks for your response! On Thu, Sep 15, 2011 at 9:22 AM, Hannes Magnusson <hannes.magnusson@gmail.com> wrote: > the change to ext/tokenizer/tests/token_get_all_variation16.phpt seems > really weird? I changed the test to not include a __halt_compiler statement anymore, as it didn't make sense in this context, and included a function call instead. The file would have been just a parse error, as __halt_compiler can only occur in the outermost scope or in namespace declaration score. Additionally the behavior of token_get_all on __halt_compiler is already tested in the other test (bug54089.phpt), so I though it wasn't necessary to test it there again. > And I'm unsure how this should actually work. > > A file containing: > <?php > > __halt_compiler() > > ?> > stuff > > Will return "stuff" after this patch.. that doesn't seem expected. First of all: The second patch (the _full one) isn't that important to me, as you can fetch the data after __halt_compiler(); quite easily in userland code, too (concat all tokens and calculate the offset from that, then substr it from the code). So if you think it doesn't make sense, I would be okay without it. The original reasoning was, that fetching the contents after __halt_compiler(); is the most probable thing you want to do if you are using token_get_all on a file with __halt_compiler. But I'm not sure that it is true and can imagine that in many cases you write scripts that don't really care about __halt_compiler statements and the additional T_INLINE_HTML would only hurt. Maybe Nicolas can provide further input on this. Thanks for looking at the patch. If you have further questions, please ask :) Nikita PS: I updated both patches to fix an issue with which tokens are considered dropped. I now explicitly state them in the if-statement, as the destroy variable wasn't quite the same (it also included the tokens that are translated into another token.)