Re: FW: Help needed in benchmarking memory patch

Salut Dmitry, Thank you for your confirmation. I also have a remark about the last_error_message i don't understand why it should live between request ? In fact it's possible that my patch wasn't very good but it's necesary to free the buffer in other place which will be executed at the end of the thread. Do you have an idea ? ts_free_thread() ? Other question, did you used my code to confirm the disabled_functions leaks ? I had found 4 other big leaks but i have to find the code which reproduce the leak We keep in touch And thanks again Michael Vergoz ----- Original Message ----- From: "Dmitry Stogov" <dmitry@zend.com> To: "'Michael Vergoz'" <mv-php@binarysec.com>; "'Andi Gutmans'" <andi@zend.com> Sent: Friday, July 14, 2006 12:35 PM Subject: RE: [PHP-DEV] FW: Help needed in benchmarking memory patch

Hi Michael, Sorry for delay. You catched real memory leaks (in ZTS mode), but your fix isn't. Seems "last_error_message" was designed to live between requests. (I'll ask authors at Monday). Also we have the same leak with "disabled_functions". I'll try prepare patch at Monday. Will you able to test it? Thanks. Dmitry.

-----Original Message----- From: Michael Vergoz [mailto:mv-php@binarysec.com] Sent: Tuesday, July 11, 2006 10:33 PM To: Andi Gutmans; 'Dmitry Stogov'; 'PHPdev' Subject: Re: [PHP-DEV] FW: Help needed in benchmarking memory patch Here is a small patch allowing to correct the small leak. This one will have to be merge with that of Michael - (its patch is correct and does not contain any leak (I were not clear in my last mail) - thus +1) http://badcode.be/~descript/PHP/PHP_5_2-php-main.diff Michael Vergoz ----- Original Message ----- From: "Michael Vergoz" <mv-php@binarysec.com> To: "Andi Gutmans" <andi@zend.com>; "'Dmitry Stogov'" <dmitry@zend.com>; "'PHPdev'" <internals@lists.php.net> Sent: Tuesday, July 11, 2006 8:29 PM Subject: Re: [PHP-DEV] FW: Help needed in benchmarking memory patch

Salut Andi & Dmitry, If you have errors in your script you have a leak in PG(last_error_message) & PG(last_error_file) because of

php_error_cb()

which allocates via strdup() these two variables. It should

be noted also

that these variables are not given with NULL after their freed in php_error_cb() - main/main.c +679 and that both last

allowance will never

be freed for each thread. They are freed with the stop of the process via

php_module_shutdown().

I propose to free these buffer in php_request_shutdown()?

In fact the

leak more are the small ones but they become repetitive in a

Threaded program.

There is also another condition to generate large mem leak

for example

by re-using a thread to carry out several script with the

continuation. There

you explodes all the records if you do not do a

ts_free_thread() just

after your php_request_shutdown(). It's for that which I

think that it

would be very judicious to add a ts_free_thread() before

each the return()

of the HttpExtensionProc() function of the ISAPI SAPI module. Michael Vergoz ----- Original Message ----- From: "Andi Gutmans" <andi@zend.com> To: "'Michael Vergoz'" <mv-php@binarysec.com>; "'Dmitry Stogov'" <dmitry@zend.com>; "'PHPdev'" <internals@lists.php.net> Sent: Tuesday, July 11, 2006 7:38 PM Subject: RE: [PHP-DEV] FW: Help needed in benchmarking memory patch

Are you sure this isn't because the IIS thread is still

running? One

thread handles multiple requests.

-----Original Message----- From: Michael Vergoz [mailto:mv-php@binarysec.com] Sent: Tuesday, July 11, 2006 10:33 AM To: Dmitry Stogov; 'Andi Gutmans'; PHPdev Subject: Re: [PHP-DEV] FW: Help needed in benchmarking

memory patch

In fact I had found some leaks it would seem that ts_free_thread() after php_request_shutdown() solves all the problem. I am checking if it's bad but if not the case the SAPI

ISAPI should

have a small modification. mv- ----- Original Message ----- From: "Dmitry Stogov" <dmitry@zend.com> To: "'Michael Vergoz'" <mv-php@binarysec.com>; "'Andi Gutmans'" <andi@zend.com>; <internals@lists.php.net> Sent: Tuesday, July 11, 2006 1:54 PM Subject: RE: [PHP-DEV] FW: Help needed in benchmarking

memory patch

Hi Michael, The patch was tested in ZTS mode too. What SAPI you are talking about? (I would like to reproduce

the problem).

Thanks. Dmitry.

-----Original Message----- From: Michael Vergoz [mailto:mv-php@binarysec.com] Sent: Tuesday, July 11, 2006 1:22 PM To: Andi Gutmans; internals@lists.php.net Subject: Re: [PHP-DEV] FW: Help needed in benchmarking memory patch Hi aAndy

We didn't find a material difference between malloc() and mmap() though, most probably because malloc() at these sizes

actually calls

mmap().

You can have good difference with the mremap() of

Linux because

it directly changes the vector into memory without recopying these data. -- Before beginning a benchmark I would like to speak about a serious problem that I have with TSRM/MM. A S1 script carried out by thread T1 will use the same memory capacity as S2 script by T2 whereas theoretically

separation is

obligatory. In addition to the problem of safety, that cause also a problem of management of the load memory of the process father. We define a father process P=while(1); When a process father starts it must initialize tsrm_startup(), sapi_startup() and php_module_startup(). At this time all the additional functions and objects are loaded and usable by all script (what is normal). With the creation of a thread one associates a TSRMLS to

it. Then one

carries out php_request_startup(), php_execute_script() and php_request_shutdown() to entirely enclose PHP/Zend. With the execution of Sx script in Tx: the variables dynamics, static and constant of this one will be stored in AG(head). The problem comes from there. It is impossible to destroy a memory which a script has just used because the process father continues to turn and nor php_request_shutdown(), php_module_shutdown(),

sapi_shutdown() and

tsrm_shutdown() will not be carried out AND THUS there will be a mem leak I am very annoyed by this problem, it is necessary

that I find a

possibility to destroy these buffer. Michael Vergoz ----- Original Message ----- From: "Andi Gutmans" <andi@zend.com> To: "'Michael Vergoz'" <mv-php@binarysec.com>; <internals@lists.php.net> Sent: Tuesday, July 11, 2006 8:26 AM Subject: RE: [PHP-DEV] FW: Help needed in benchmarking

memory patch

Sorry I didn't make clear that only the malloc()/free()

part works.

The mmap() and Win32 Allocation code is commented. We didn't find a material difference between malloc() and

mmap() though,

most probably because malloc() at these sizes actually

calls mmap().

So really most important is for some ppl to run some

benchmarks on this.

Thx. Andi

-----Original Message----- From: Michael Vergoz [mailto:mv-php@binarysec.com] Sent: Monday, July 10, 2006 5:25 PM To: Andi Gutmans; internals@lists.php.net Subject: Re: [PHP-DEV] FW: Help needed in benchmarking

memory patch

Hi Andi, I appreciated to see this code which I must still

study. Already a

small remark it is that the system call mremap() is not

portable it

is available only on Linux. Then the size of the segments

must always

"be paginated" - normally your code is good on this

level. I

think that a concept of kernel-memory and

user-memory can be

very interesting to study, especially in threads

environments

- that I use. I would continue to look at your code but now I will

sleep (04:26

pm)! mv- ----- Original Message ----- From: "Andi Gutmans" <andi@zend.com> To: "'Andi Gutmans'" <andi@zend.com>; <internals@lists.php.net> Sent: Monday, July 10, 2006 5:57 PM Subject: RE: [PHP-DEV] FW: Help needed in benchmarking

memory patch

No help??? Come on guys. I'm sure some of you can spare a

few idle cycles

:)

-----Original Message----- From: Andi Gutmans [mailto:andi@zend.com] Sent: Friday, July 07, 2006 8:38 PM To: internals@lists.php.net Subject: [PHP-DEV] FW: Help needed in benchmarking

memory patch

Sending 3rd time this time without the

attachment. You can

get the diff at http://gutmans.org/alloc.zip -----Original Message----- From: Andi Gutmans [mailto:andi@zend.com] Sent: Thursday, July 06, 2006 11:06 PM To: 'internals@lists.php.net' Subject: Help needed in benchmarking memory patch Hi all, Attached is a patch we've been working on to improve

the memory

management of PHP. This patch's main advantage is

reducing the

memory footprint of PHP, and therefore allowing

it to scale

better. Although at low concurrencies the change

seems to

be negligble, while testing at higher

concurrencies we saw

significant improvement with this patch, mainly due to lower memory usage. We would very much appreciate if people here tested

this patch and

send in their results. It would help us understand if

others are

getting consistent results with ours, and also see

whether there

are any additional improvements we should make. This patch should apply cleanly to the PHP 5.2 CVS

tree. There's

not much tuning that needs to be done. It uses malloc() to allocate large memory blocks. You can

control the size

of these blocks by setting the ZEND_MM_SEG_SIZE

environment

variable before starting Apache/PHP. You may use K

or M to play

around with the block size. The default we are using is 256KB which seems to be a good balance. Any feedback would be appreciated! Andi -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php

-- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php

-- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php

-- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php

-- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php