Re: Segfault with tokenizer on Win32

Forget that one it didnt help :) - the workaround kind of proves that it's not zend Another work around - guarenteed to work 100% ! <cheesy workaround> $fh = popen(dirname(__FILE__)."/tokenizer_serialize.php $filename","r");

       $this->tokens= unserialize(fread($fh,filesize($filename) * 16));
       pclose($fh);

-------------------------------tokenizer_serialize.php------------------------------- #!/usr/bin/php -q <? dl('tokenizer.so'); $filename= $argv[1]; $contents = implode('',file($filename));

       $tokens = token_get_all($contents);

print serialize($tokens); ?> ----------------------------------------------------- </cheesy workaround> regards alan Alan Knowles wrote:

The definitive work around : in Zend/zend_alloc.c #define ZEND_DISABLE_MEMORY_CACHE 1 After staring at the tokenizer code for quite a while - I cant see anything wrong with it .... back to the original suspision about the zend cache...... does this affect ZE2 as well? regards alan Sebastian Bergmann wrote:

Alan Knowles wrote:

theres a work around in there (basically you cant re assign $this->tokens = token_get_all() ) you need to do something like $this->tokens_all[$file] = token_get_all($file); $this->tokens =& $this->tokens_all[$file]

Crashes here as well. I just noticed, that Andrei's sample (php4/ext/tokenizer/tokenizer.php) crashes as well. It didn't crash with CVS from some days ago.