Re: [PATCH] Scanner "diet" with fixes, etc.

Arnaud Le Blanc wrote:

Hi, On Mon, May 4, 2009 at 9:36 AM, shire<shire@php.net> wrote:

Regarding the ZEND_MMAP_AHEAD issue and the temp. fix that Dmitry put in we need to find a solution to that, perhaps I can play with that this week too as I think I'm seeing some related issues in my testing of 5.3. Essentially we abuse ZEND_MMAP_AHEAD by adding it to the file size and passing it to the mmap call which isn't at all valid and only really works up to PAGESIZE. We could possibly use YYFILL to re-allocate more space as necessary past the end of file to fix this.

I was thinking of doing something like that with YYFILL too. However there is a bunch of pointers to take in to account and to update (e.g. yy_marker, yy_text, etc).

Yeah, I'm pretty sure that's how most of the example re2c code is setup: #define YYFILL(n) {cursor = fill(s, cursor);} uchar *fill(Scanner *s, uchar *cursor){

    if(!s->eof){

unint cnt = s->lim - s->tok; uchar *buf = malloc((cnt + 1)*sizeof(uchar)); memcpy(buf, s->tok, cnt); cursor = &buf[cursor - s->tok]; s->pos = &buf[s->pos - s->tok]; s->ptr = &buf[s->ptr - s->tok]; s->lim = &buf[cnt]; s->eof = s->lim; *(s->eof)++ = '\n'; s->tok = buf;

    }
    return cursor;

} -shire