Re: Re: [Fwd: Re: PHP memory fragmentation]

Hi Andrew, This patch looks interesting. Fragmentation and the odd "cron job" that consumes lots of memory is certainly a problem that can affect many. Main question is wether this only affects FastCGI. I think it is also relevant at least to Apache pre-fork and it might be interesting to support that as well, possibly in a generic way. Anyone have any thoughts regarding this issue? Andi At 01:49 PM 6/24/2005 -0700, Andrew Prendergast wrote:

Just to add, with the patch one can set PHP_FCGI_MAX_REQUESTS quite high, which will give a big reqs/sec performance boost, eg: PHP_FCGI_CHILDREN=10 PHP_FCGI_MAX_RAM_INCREASE=25 PHP_FCGI_MAX_REQUESTS=20000 ap. On Fri, June 24, 2005 13:15, Andrew Prendergast said: Resend. List serv problems. ap. ---------------------------- Original Message ---------------------------- Subject: Re: PHP memory fragmentation

From:    "Andrew Prendergast" <ap@tellusion.com>
Date:    Sun, June 12, 2005 21:09
To:      internals@lists.php.net

-------------------------------------------------------------------------- Please find attached a diff against HEAD & a modified version of the file implementing my proposed PHP_FCGI_MAX_RAM_MB & PHP_FCGI_MAX_RAM_INCREASE changes. Example invocation of PHP with 10 child processes that are automatically restarted if they increase in size by more than 25%: PHP_FCGI_CHILDREN=10 PHP_FCGI_MAX_RAM_INCREASE=25 PHP_FCGI_MAX_REQUESTS=20 export PHP_FCGI_CHILDREN PHP_FCGI_MAX_REQUESTS PHP_FCGI_MAX_RAM_INCREASE /usr/local/zeus/web/bin/fcgirunner \ --user=www --group=www 8002 \ /usr/src/usr.local/php-5.0.4/sapi/cgi/php -c /usr/local/lib/php.ini Compile with -DDEBUG_FASTCGI to see whats going on. I've been testing it for the last 24 hours with various volumes of data and it definitely does the trick. Regards, ap. On Sat, June 11, 2005 18:36, Andrew Prendergast said:

After a bit more investigation, I would like to change this paragraph: Once a script finishes executing, although the garbage collector

technically has freed the ram, the pages are still allocated. The next time the thread executes, there is a whole pile of swapping and if a memory hungry script gets run again, the memory footprint increases.

To: Once a script finishes executing, although the garbage collector

technically has freed the ram, the pages are still allocated. If the thread is re-used, even for a small script, much of the previously swapped out ram is swapped back in due to fragmentation. The memory footprint of the thread will increase further if a different memory hungry script is executed, or a small script that has a different allocation pattern to the first memory eating script is executed.

Regards, ap. On Sat, June 11, 2005 16:30, Andrew Prendergast said:

Hey Folks, I'm not on the list so please reply back to my email. I have been having some problems with memory fragmentation in PHP when

working with large data structures.

Once a script finishes executing, although the garbage collector

technically has freed the ram, the pages are still allocated. The next time the thread executes, there is a whole pile of swapping and if a memory hungry script gets run again, the memory footprint increases.

Now consider what happens when I have 20 processes in an FCGI pool,

each one getting above 300MB in size, its not good.

My solution which I have implemented as part of the FCGI SAPI (thanks

to GeorgeS, hartmut, Pierre & Derick for their guidance) is the

introduction of two new environment variables, PHP_FCGI_MAX_RAM_MB & PHP_FCGI_MAX_RAM_INCREASE. PHP_FCGI_MAX_RAM_MB allows one to set a limit (in megabytes) of how large an FCGI process is allowed to grow to. PHP_FCGI_MAX_RAM_INCREASE allows one to set a maximum % increase compared to when the process first started. If either of these limits are exceeded, the process will be restarted

AFTER script execution has completed (unlike the --enable-memory-limit stuff).

Please get back to me on: 1. your thoughts on this 2. an idea of how big an average PHP process should be (mine are 150MB

when they start, 160MB after running some average scripts, and grow to above 300 MB when working with big data structures and complex object caching code)

3. how do i get this back into the PHP source tree! (sorry newbie) Regards, ap.

-- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php