Re: [PEPr] Comment on PHP::Lexer

bertrand Gugger wrote:

I would also like to recurse the token within themselves. à+

I agree that this would be nice, but it can't be done with regular expressions. He needs to write a finite state machine to support this. Also, I'd be careful about what you're doing with it. The purpose of a lexer is to just simply the input for a parser. The parse then converts the token stream into a syntax tree as defined by a context free grammar. That's where it absolutely necessary to have a finite state machine. -Nikolas 'Atrus' Coukouma