Re: [PATCH] Some string changes/optimizations

A bit of a messy patch in that it doesn't have a single top-level directory, but after hacking it, it applied. For others testing this, make sure you run ext/tokenizer/tokenizer_data_gen.sh and of course genfiles with the latest re2c. I'm still not getting something that works though. Everything compiles fine, but make test is throwing: Fatal error: Invalid opcode 56/16/8. in /Users/rasmus/php53/run-tests.php on line 547 Could we get a cleaner patch to have a look at? One that doesn't hurt my brain quite as much to get working? -Rasmus Matt Wilmas wrote:

Hi all, This patch... Saves one opcode per interpolated string by moving what was the first ADD_STRING/ADD_VAR op into INIT_STRING; changes the ADD_[CHAR|STRING|VAR] op types from (TMP, <whatever>) to (<whatever>, UNUSED), since that seems to make more sense (see ADD_ARRAY_ELEMENT) and simplifies things a bit, and saves some znode copying while compiling. Next, after nowdoc support was added, I noticed it was mostly duplicate heredoc code in the scanner, so I combined them for the most part, removing the NOWDOC tokens, etc. Is that OK? It seems like they may have just been there for the parser... but I updated the parser so that static heredocs also work like nowdocs. Also removed the ST_START_[HEREDOC|NOWDOC] states in the scanner, by doing their work (to catch immediate ending label) in the initial heredoc rule. While removing the NOWDOC references from zend_highlight.c, I made a little change that I think improves (err... adds) highlighting of variables in double-quoted strings, and makes literal text in heredocs/backticks the correct "highlight_string" color. This makes coloring consistent across all strings. :-) Before/after example: http://realplain.com/php/highlight.html Speaking of backticks, while updating their parser grammar, they now behave more precisely like shell_exec(), in that cmd (constant string) or $cmd (one variable) won't use INIT_STRING and create a temporary variable... Oh, almost forgot a couple things -- for HEAD: updated the yyless() definition to match 5_3 (Nuno didn't get to it yet...), and moved HANDLE_NEWLINES() so nowdoc text is copied first, otherwise the line number would be off if there's an error in zend_copy_scanner_string. For 5_3: there are NOWDOC references in tokenizer_data.c that the patch doesn't remove, as I think it's supposed to be automated, though they're missing in HEAD. http://realplain.com/php/string_optimizations.diff http://realplain.com/php/string_optimizations_5_3.diff Thanks, Matt P.S. (for Marcus?), the ({LABEL}|([']{LABEL}['])|(["]{LABEL}["])) part of the heredoc rule crashed re2c (Windows) until I added the extra ( ), which shouldn't be needed AFAIK and was fine with just 2 alternate matches...