Re: Moving to an AST-based parsing/compilation process

On 09/06/2012 11:52 AM, Ivan Enderlin @ Hoa wrote:

Hi Dmitry, On 06/09/12 07:37, Dmitry Stogov wrote:

Hi Nikita, Personally, I don't see any reason to build AST. As you mentioned yourself, it will be slower and will require more memory. On the other hand AST itself would allow to perform only very basic optimizations. Most of them can be easily done on VM opcode level as well.

The lexing and parsing processes will not be slower than actual, and the construction of an AST is a new process. Well, as usual, new process requires new resources. But if we look further, it will certainly be a nice tool to perform better opcode caching, it will remove a lot of hacks, it will allow third-part tools working on safeness, security, quality etc. to go deeper at low-costs which is very important for PHP community, it will facilitate future works (implementations, features etc.) for PHP… Moreover, it may be possible that compiler compilers have a better lexing and parsing processes than actual?

Few years ago we replaced flex with re2c, and got some speedup because re2c generated scanner used mmap() and didn't check for end of buffer, but after a while we realized that in case the script size is multiplication of PAGE_SIZE the scanner just crashes, so we had to add hacks to workaround :)

Someone said that AST is more “academical”, yes it is and that is why we can benefit from already done researches in this area to avoid memory overhead (one among others). We are not the first ones facing this problem. It requires some researches before starting to develop this. Let's try as a POC and we will quickly see if this is a wrong way or not.

Of course you can try it and in case it's better, faster and/or more clear it may be accepted. Currently, PHP allows overriding of "zend_compile" routine, so as a first step you even may implement it as a PHP extension without ZE modification. Thanks. Dmitry.

Cheers.

Also, as it's not an easy task, the old "ugly hacks" will be replaced with new mistakes, which would require new "hacks" in the future :) The only real advantage could be an ability to expose AST to PHP scripts, but only few people may need it. Thanks. Dmitry. On 09/04/2012 11:57 PM, Nikita Popov wrote:

Hey folks! Some people asked me what the advantages of using an AST-based parsing/compilation process are, so I put together a few quick notes in an RFC: https://wiki.php.net/rfc/ast_based_parsing_compilation_process It would be nice to get a few comments from other core devs on this. Nikita