I consider it obvious why it makes no sense to abstract the scanner input of the engine, and I guess this is not very good - since some of you may not understand what it is about.
The reason it makes no sense is very simple. The scanner Sascha wrote doesn't behave in a different way than the current scanner. Nor does it have any drawbacks when compared to the flex scanner. It's a replacement, that performs better and is more compatible than the C++ based flex scanner.
Now, let's imagine someone sent us a new implementation of the DOM-XML module, which has an identical API to the last bit (perhaps with some additions), performs faster, and is more compatible. Would we add DOM-XML-NG, and let people choose? Of course not, because it's a plain dumb thing to do. The old DOM-XML extension will be removed and the new one would take its place.
The scanner case is similar, except it's a much more fundamental component, which makes even *less* sense to abstract. Abstracting it gives *nothing* from a technical point of view. The single reason Sascha did that, was because he is not happy with the Zend Engine license, and doesn't want to submit it the way it is, to make a point.
I've been discussing the Zend Engine license with the 'leaders' of the German PHP community on Thursday, and with members of the community and the PHP Group on Friday. As mentioned there, the Zend Engine license is being reviewed, and may change in the next few months. Especially in the light of this, I see Sascha's changes as making even less sense than what they would have made before, and that wasn't too much to begin with. Abstracting the scanner buys you *nothing* from a technical point of view.
Zeev