Re: PHP_Parser class - any plans?

have a look at this, it is a 'phpdoc like' program written using the tokenizer. http://docs.akbkhome.com/PHPcodedoc/classes/PHP_CodeDoc.html#_parse_file -- note that it has to use a this kludge to actually do the parsing as currently the tokenizer has a memory allocation problem which prevents it parsing more than a few files without segfaulting.... $fh = popen(dirname(__FILE__)."/tokenizer_serialize.php $filename","r");

tokenizer_serialize.php #!/usr/bin/php -q <? dl('tokenizer.so'); $filename= $argv[1]; $contents = implode('',file($filename));

       $tokens = token_get_all($contents);

print serialize($tokens); ?> the return value from token_get_all() is an array of tokens, each token is either a string or an array, if it is an array, the first value is the 'keyword type id', eg. T_STRING,T_FUNCTION etc. you can use token_name($v[0]) to find out these values. regards alan Lux wrote:

Are there any docs at all on this tokenizer extension? I will take a look at the source when I get a chance at least. My C skills are practically nonexistant, so I won't be able to stabilize it or anything, but I'm still rather intrigued by this idea. I agree with Stig that it would be a waste to implement this in PHP space, since extensions already have access to an existing low level parser, and since each time the Zend engine changes, the PHP version would require constant updating to keep in sync, and it would be probably be really slow. Still, something that offered higher level insight into a lower level extension that does the grunt work could take the idea into new directions as well. Lux On Thu, 2002-05-16 at 13:00, Sebastian Bergmann wrote:

Simon Willison wrote:

Are there any plans to put together a PEAR tool for parsing and "understanding" PHP scripts?

ext/tokenizer is the tool of choice for such an undertaking. Or would be, if it were stable. -- Sebastian Bergmann

 http://sebastian-bergmann.de/                 http://phpOpenTracker.de/

Did I help you? Consider a gift: http://wishlist.sebastian-bergmann.de/ -- PEAR Development Mailing List (http://pear.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php