Re: Segfault with tokenizer on Win32

Have a look at the bug report for it - http://bugs.php.net/bug.php?id=16939 theres a work around in there (basically you cant re assign $this->tokens = token_get_all() ) you need to do something like $this->tokens_all[$file] = token_get_all($file); $this->tokens =& $this->tokens_all[$file] Andrei knows about it, I'm not sure what his schedule is like for fixing it is though.... this might save you a bit of work -copy and paste at will :) http://docs.akbkhome.com/phpmole/classes/php_generator.html regards alan Sebastian Bergmann wrote:

Does anyone have an idea why the following code crashes on Win32 in Release_TS mode, but not in Debug_TS mode (hence no stacktrace)? <?php class CodeParser {

     var $tokens = array();
        function CodeParser($file = '') {
       if (!empty($file)) {
         $this->parse($file);
       }
     }
        function parse($file) {
       $this->tokens = token_get_all(
         implode(
           '',
           file($file)
         )
       );
     }
        function getSource() {
       $source = '';
       $state  = 0;
          foreach ($this->tokens as $token) {
       	if (is_array($token)) {
   		    if ($state == 1 && $token[0] == T_STRING) {
   			    $token[1] = preg_replace(
   			      '!([a-z])([A-Z])!e',
   			      '"$1_".strtolower("$2")',
   			      $token[1]
   			    );
          			$state = 0;
   		    }
              else if ($token[0] == T_FUNCTION) {
   			    $state = 1;
   		    }
          		$buffer = $token[1];
   	    } else {
   		    $buffer = $token;
   	    }
          	$source .= $buffer;
       }
          return $source;
     }
        function getTokens() {
       return $this->tokens;
     }

} ?>