Re: [PHP4BETA] PHP4b3 RC3 Memory usage

Ok done the test rounds again ... here's the results: No changes to the source code startup 1776 (this seems to be a very stable factor :P) rb scripts 7340 With the php_request_shutdown + initial_array (8) changes, startup 1776 rb scripts 6644 Now with the MAX_CACHED items changed to half there original value (eg mem 32, entries 8) (so this is both the request_shutdown + CACHED items + INITIAL_ARRAY on 8 + --disable-debug + --disable-inline). startup 1776 rb scripts 6584 So were edging in the right direction with this ... but we are making small steps, not leaps .. im afraid its gonna take another meg or so in memory usage before its up to par with php3 ;/ Now to go buckwild for a second ive changed the cached entries to 128 / 64, to do some performance ratings on these changes... (also reverted request_shutdown in this test). startup 1780 rb scripts 7412 So it would seem the cost of those entries is about a 100 kbytes for a 50 % < 100% > 200 % change. ergo, setting them to 16 / 4 would presumable cause the memory footprint to be around 6484 kbytes. The Number of viewings per second on the RB scripts hardly changes by these settings, which is plausable since there's a lot of memory adressing going on, and very heavy MySql usage, one could say those are prolly the limiting factors there (and high amount of calls in one page request). On the other hand if i try to benchmark the info.html and simplequery.html pages, the results are more extravegant. (disclaimer: this is running on a p iii 500, 256 mb ram, scsi ii uw, but while running x-windows etc, so results are well tested and stable, but not representative for real life php performance. The point im trying to make is a procentual one, and not a bottom line req/sec one :P) With the [cached mem = 128, entries = 64] info.html : Requests per second: 205.85 (Transfer rate: 5333.15 kb/s received!) simpleq.html: Requests per second: 288.35 Now with [cached mem = 32, entries = 8) info.html: Requests per second: 213.29 simpleq.html Requests per second: 334.84 So the performance of /simple/ scripts actualy seem to improve when reducing the caching (overhead?). Now to benchmark cached mem = 32, entries = 8, and php_request_shutdown(....,1) (note: cause of this alocation scheme, the memory usage noticibly jumps up and down between 1600 and 2100 all the time) info.html Requests per second: 48.91 simpleq.html Requests per second: 48.90 So i think that despite the sounds-good factor of the request_shutdown(..,1), it gives the php engine such a performance hit on small scripts, that we would rather take the small memory loss i presume .. the difference is just to big (500% to 600% speed decrease!). to make sure that the request_shutdown and the smaller cached mem/entries were not the lethal combination, ive also tested the situation with cached mem set to 128, and entries to 64, and request_shutdown set to 1.. info.html Requests per second: 47.90 simpleq.html Requests per second: 48.36 So my conclusion so far? i gues the request_shutdown stays set to 0 on my box, and the cached mem/entries prolly set to 32 / 8 ... it makes enough of a mem difference, without impacting on performance substancialy. But then again ive only tested a small number of situations ( <? phpinfo(); ?>, a simple mysql query script, and a very large complex reviewboard code base). It might be that the performance impact on highly math based scripts etc might be different from this situation. If anyone would like me to test these, feel free to ask :) Ps to give ya'll some insight in my test envirioment, ive included some of the simple scripts im using for these builds / benchmarks: build3 : builds my php3 envirioment (config + make + changes config files + restarts server) build4: builds my php4 envirioment ( ... ) info.html (duh) simpleq.html (does a simple count(*) on a DB, en fetches single result) -- Chris Chabot CTO Reviewboard Magazine Andi Gutmans wrote: > In main.c line 771 (php_request_shutdown function) change the second > parameter 0 to a 1. What happens is, we made a change so that memory is > cached also between requests and not only for the current request, this > change will make it clean the cache each request. This is better for > performance but eventually requires more memory. It would change each > individuals requests' memory usage though. You can also play around with > MAX_CACHED_MEMORY and MAX_CACHED_ENTRIES in zend_alloc.h if you want which > control what the maximum size of cached memory is and how many blocks of > each size to save. > > Andi > > At 12:37 AM 11/11/99 +0100, Chris Chabot wrote: > >Ok made the modification as requested, results are: > >(apache 1.3.9/php4b3 rc4 as described below) > > > >start reviewboard scripts (after 20000 requests) > >1764 7180 > > > > > > > >So it helped a little bit, but not enough to make a huge difference to the > >problem > >as described below.. which makes sence since only one of the global > >variables is > >an array .. The thing that freaks me out is that its also not a memory leak > >either ... the memory used stabalises after about 500 hits and doesnt grow > >after > >that (tried this upto 5,000,000 hits during a couple of hours :P). So there's > >something about either the size of the code base (its a rather large > >project base, > >eg 340Kb of php script, 8000 lines of code +), or some other unforseen > >thing i do > >in these scripts that freak it out, that worked fine under the 3.x series.... > > > >again my offer is open for people (that i can trust) to play around w/ the > >code > >base on this box. But since it is a product of the reviewboard magazine, > >and fully > >tm'd and C'd, i couldnt send it to ya'll... > > > > > >Anyways hope im able to help out a bit and that we can get this problem solved > >before php4 final would hit the shelves :) > > > > -- Chris Chabot > > > > > >Andi Gutmans wrote: > > > > > Try and edit zend_compile.h and change #define INITIAL_OP_ARRAY_SIZE 64 to > > > 8. It might help a little bit. Let us know of the results. > > > > > > Andi > > > > > > At 01:13 PM 11/10/99 +0100, Chris Chabot wrote: > > > >Ok ive finaly come around to setting up some test cases. > > > > > > > > These are the 2 different configurations: > > > > > > > > apache 1.3.9 / php 3.0.12 > > > > > > > > php conf: > > > >./configure --disable-inline --disable-debug --enable-sysvsem > > > >--enable-sysvshm \ > > > >--enable-track-vars --with-zlib > > > >--with-mysql=/usr/local --with-gettext --with-gd \ > > > >--with-ttf --with-apache=../apache_1.3.9 > > > > > > > > CFLAGS=-O2 > > > > > > > >And > > > > Apache 1.3.9 / php 4b3 RC4 > > > > > > > > php conf: > > > > > > > >./configure --disable-inline --disable-debug --enable-sysvsem \ > > > > --enable-sysvshm --enable-track-vars --with-zlib > > --with-mysql=/usr/local \ > > > > --with-gettext --with-gd --with-ttf --with-apache=../apache_1.3.9 \ > > > > --with-readline --with-ftp --with-mcrypt --with-mhash > > > > > > > > CFLAGS=-O2 > > > > > > > > > > > >Recorded are 3 states. eg startup mem size, mem size after hitting the <? > > > >phpinfo(); ?> page a few times (50k times), and after doing a simple mysql > > > >query script (that basicly > > > >does a count(*) from table, fetches result in a list($count) = > > > >mysql_fetch_row($res) statement, and echo's $count. > > > > > > > >I also included apache bench's requests a second result, to give some > > > >insight into php4's normal performance. given the nature of the scripts, > > > >they shouldnt differ /to/ much since > > > >there both very small scripts where I/O gives more overhead then the > > > >scripting envirioment. > > > > > > > > > > > > Start info > > query > > > > > > > >php3 1500 1868 (no leaks) 1948 (no > > leaks) > > > >reqs/sec 361.41 349.14 > > > > > > > >php4 1776 2128 (no leaks) 2208 > > > >reqs/sec 225.99 343.11 > > > > > > > > > > > >As you can see php4 is performing a little slower on real simple scripts. > > > >but its memory usage on the info.html and simplequery.html pages is very > > > >respectable. A little higher, > > > >ok, but nothing to be upset about. > > > > > > > >What strikes me weird here is that the info page is so much slower, but i > > > >gues php4 just has a lot more cool modules to dump in the info page. > > > > > > > >Now to do this on the reviewboard code (quite complex, more then 100 > > > >variables , and 243 functions in global scope. > > > >(20k requests) > > > > > > > >php3 5088 (no leaks) > > > > > > > >php4 starts at 6208 then grows to around 7340 ... keeps hanging > > > >around that number. > > > > > > > >As u can see, there's no memory footprint problem with the basic > > > >configuration, or basic scripts. But as soon as i unleash some real life > > > >big scripts @ it (i tried this w/ other > > > >script bases to, same results) the memory usage is substancialy higher > > > >then w/ php 3 ... 3 1/3 megs per process extra is a killer for a box with > > > >100 httpd's open ... thats 300 > > > >megs of memory dude! :) this would definatly scare me away from an > > upgrade.... > > > > > > > >if there's any other things i can try, or even give Sascha or rasmus > > > >access to one of my test boxes to they can play with it (cable modem or t1 > > > >connects, dont wurry) let me know. > > > > > > > > -- Chris Chabot > > > > > > > > CTO Reviewboard Magazine > > > > > > > > > > > >Sascha Schumann wrote: > > > > > > > > > On Tue, Nov 09, 1999 at 09:57:42AM -0500, Rasmus Lerdorf wrote: > > > > > > This memory usage issue is definitely something we need to > > address. The > > > > > > Zend parser should be much more efficient with its memory usage > > > > because of > > > > > > the reference counting, and the various extensions should at worst be > > > > > > using as much memory as PHP 3, so going from 3M to 7M per httpd is a > > > > > > mystery to me. I think we need to do a bit of scientific testing > > > > > > here. Have a simple little script that just does a "Hello > > > > > > World" to > > > > start > > > > > > with and bang at it with ab or http_load with both PHP 3 and 4. > > > > > > > > > > > > Then make the script do a simple MySQL query and compare the numbers. > > > > > > > > > > > > I'll try to find some time today to do a check on my box to see > > if I can > > > > > > verify your numbers. > > > > > > > > > > I've just sent the attached email to Chris. The page I > > > > > accessed contained only <?phpinfo()?>, so there is a chance > > > > > that the leak appears only in complex scripts. > > > > > > > > > > -- > > > > > > > > > > Regards, > > > > > > > > > > Sascha Schumann > > > > > Consultant > > > > > > > > > > > > ------------------------------------------------------------------------ > > > > > > > > > > Subject: Re: [PHP4BETA] PHP 4.0b3, second try > > > > > Date: Tue, 9 Nov 1999 16:10:48 +0100 > > > > > From: Sascha Schumann <sascha@schumann.cx> > > > > > To: Chris Chabot <chabotc@reviewboard.com> > > > > > References: <Pine.LNX.4.04.9911091245580.22365-100000@tjafs.fast.no> > > > > <38282CB8.CBE01F77@reviewboard.com> > > > > <19991109151912.A16464@schumann.cx> > > > > <382832A7.43210EA0@reviewboard.com> > > > > > > > > > > > ps testing tool i use to test these releases is http_load > > > > > > (http://www.acme.com/software/http_load/) > > > > > > i strongly sugest this program to anyone who wants to do some serious > > > > load > > > > > > testing. > > > > > > > > > > How large is the process at startup time? If it grows > > > > > drastically during http_load, you probably have found a > > > > > memory leak. > > > > > > > > > > I've compiled 4.0b3-RC3 with > > > > > > > > > > CFLAGS="-O2 -march=pentiumpro" \ > > > > > ./configure \ > > > > > --disable-debug \ > > > > > --disable-inline \ > > > > > --with-apache=../apache_1.3.9 \ > > > > > --with-mysql=/local/mysql > > > > > > > > > > Apache was compiled then with > > > > > > > > > > CFLAGS="-O2 -march=pentiumpro" \ > > > > > ./configure \ > > > > > --prefix=/some/where \ > > > > > --activate-module=src/modules/php4/libphp4.a > > > > > > > > > > Now, on startup the process is a little bit larger than 2MB. > > > > > That includes the complete PHP module. After running 70 > > > > > parallel HTTP queries against a PHP-generated page, the > > > > > server process hardly grew in size (+70KB). > > > > > > > > > > What does your demo page(s) do? Is the effect similar, if the > > > > > only page you access is "<?phpinfo()?>" ? > > > > > > > > > > -- > > > > > > > > > > Regards, > > > > > > > > > > Sascha Schumann > > > > > Consultant > > > > > > > > > > > > ------------------------------------------------------------------------ > > > > > -- > > > > > PHP 4.0 Beta Mailing List > > > > > <http://www.php.net/version4/> > > > > > To unsubscribe, e-mail: php4beta-unsubscribe@lists.php.net > > > > > For additional commands, e-mail: php4beta-help@lists.php.net > > > > > To contact the list administrators, e-mail: > > php4beta-admin@lists.php.net > > > > > > > > > > > >-- > > > >PHP 4.0 Beta Mailing List > > > ><http://www.php.net/version4/> > > > >To unsubscribe, e-mail: php4beta-unsubscribe@lists.php.net > > > >For additional commands, e-mail: php4beta-help@lists.php.net > > > >To contact the list administrators, e-mail: php4beta-admin@lists.php.net > > > > > > --- > > > Andi Gutmans <andi@zend.com> > > > http://www.zend.com/ > > > > > > -- > > > PHP 4.0 Beta Mailing List > > > <http://www.php.net/version4/> > > > To unsubscribe, e-mail: php4beta-unsubscribe@lists.php.net > > > For additional commands, e-mail: php4beta-help@lists.php.net > > > To contact the list administrators, e-mail: php4beta-admin@lists.php.net > > --- > Andi Gutmans <andi@zend.com> > http://www.zend.com/