Bug #61558 [Com]: Runaway spawning of children after pipe error

Edit report at https://bugs.php.net/bug.php?id=61558&edit=1 ID: 61558 Comment by: jordansmith6532 at gmail dot com Reported by: phpbug2012 at tgabber dot mine dot nu Summary: Runaway spawning of children after pipe error Status: Analyzed Type: Bug Package: FPM related Operating System: Debian Linux PHP Version: 5.3.10 Block user comment: N Private report: N New Comment: I simply couldn't resist praising the way you play with words. This is a perfect example of a well-written blog post. If you are looking for a dedicated team of editing and proofreading experts to polish your academic write up, the Livewebtutors platform can be the one-stop solution for you. The portal has comprehensive Assignment Help services ranging right from the writing of short essays to tackling be complicated dissertation writing process. If you are want to see a remarkable transformation in your academic grades, this is the platform for you. visit:- https://www.livewebtutors.com/canada Previous Comments: ------------------------------------------------------------------------ [2015-08-03 03:25:28] kobenews at cox dot net I think this issue is related to a bug I just posted. Same behavior where php-fpm restarts the process over and over again, causing php-fpm master to use 100% cpu. The test script is a little bit simpler, though requires exec() and running a SSH command using multiplexing. https://bugs.php.net/bug.php?id=70185&edit=2 ------------------------------------------------------------------------ [2014-07-25 13:07:22] Danack at basereality dot com I'd recommend being inspired by aka copying the behaviour of Supervisord. http://supervisord.org/subprocess.html Basically it does what it probably the best behaviour of: i) If child processes exist too quickly, put them in a 'back off' state, to rate limit the number of processes attempting to be started. ii) If they still fail to start after a reasonable number of retries stop trying to start it for now. Admittedly, sounds like a big task for something that should be a rare event. ------------------------------------------------------------------------ [2014-07-25 09:06:12] tony2001@php.net Ok, so FPM basically starts the children, but they die immediately (for a reason) and FPM master process starts the new ones. I can think of adding some option that would limit the number of children spawned per second, or probably abort the whole process if N children were created in the last N seconds, but that kind of solution looks quite hacky to me. ------------------------------------------------------------------------ [2014-07-24 19:44:00] Danack at basereality dot com I think there is an easier way to trigger this (and probably something that I'll open as a separate bug): i) Include a library function in an extension. ii) Don't include the library in the linking step for php-fpm. iii) Watch the error log fill up with: [24-Jul-2014 18:58:27] NOTICE: [pool www] child 22710 started [24-Jul-2014 18:58:27] WARNING: [pool www] child 22710 exited with code 127 after 0.044963 seconds from start [24-Jul-2014 18:58:27] WARNING: [pool www] child 22710 said into stderr: "php-fpm: pool www: symbol lookup error: /usr/local/lib/php/extensions/no-debug-zts-20131226/imagick.so: undefined symbol: GetMagickVersion" Once this happens you need to kill -9 the php-fpm master process to stop it spawning pool workers. Obviously there shouldn't be errors like this in an extension, but also PHP-FPM shouldn't become unresponsive and start spawning workers like crazy. ------------------------------------------------------------------------ [2013-05-23 13:44:24] pavel at stack dot ee PHP 5.3.23 with Suhosin-Patch (cli) (built: Mar 26 2013 14:07:09) Copyright (c) 1997-2013 The PHP Group Zend Engine v2.3.0, Copyright (c) 1998-2013 Zend Technologies FPM POOL CONFIG: [storage] listen = /tmp/fpm_storage.sock listen.owner = storage listen.group = storage listen.mode = 0666 user = storage group = storage pm = dynamic pm.max_children = 50 pm.start_servers = 10 pm.min_spare_servers = 5 pm.max_spare_servers = 35 NGINX: location ~ \.php$ { fastcgi_pass unix:/tmp/fpm_storage.sock; fastcgi_index index.php; include fastcgi_params; fastcgi_param DEVENV on; fastcgi_intercept_errors on; fastcgi_ignore_client_abort off; fastcgi_connect_timeout 60; fastcgi_send_timeout 180; fastcgi_read_timeout 180; fastcgi_buffer_size 128k; fastcgi_buffers 4 256k; fastcgi_busy_buffers_size 256k; fastcgi_temp_file_write_size 256k; } ~/.ssh/config: Host * ConnectTimeout 2 TCPKeepAlive yes Port 22 Identityfile ~/.ssh/server1000 ControlMaster no ControlPath ~/.ssh/master-%r@%h:%p Host server1000 Hostname 192.168.2.2 Creating connection to remote server: # ssh -o "ServerAliveInterval 15" -o "ServerAliveCountMax 1" -MN user@server1000 # ls -la ~/.ssh/ srw------- 1 storage storage 0 Mar 26 15:46 master-root@192.168.2.2:22 # ssh server1000 whoami root and this simple script for which was run from browser and i'm getting same behaviour in Debian and FreeBSD <?php for ($i = 0; $i < 100; ++$i) { echo exec("ssh root@192.168.2.2 ls -la"); } ?> [26-Mar-2013 15:57:00] NOTICE: configuration file /etc/php5/fpm/php-fpm.conf test is successful [26-Mar-2013 15:57:00] NOTICE: fpm is running, pid 20053 [26-Mar-2013 15:57:00] NOTICE: ready to handle connections [26-Mar-2013 15:58:33] NOTICE: [pool storage] child 20054 exited with code 0 after 92.745193 seconds from start [26-Mar-2013 15:58:33] NOTICE: [pool storage] child 20324 started [26-Mar-2013 15:58:33] NOTICE: [pool storage] child 20324 exited with code 0 after 0.005245 seconds from start [26-Mar-2013 15:58:33] NOTICE: [pool storage] child 20325 started [26-Mar-2013 15:58:33] NOTICE: [pool storage] child 20325 exited with code 0 after 0.005121 seconds from start [26-Mar-2013 15:58:33] NOTICE: [pool storage] child 20326 started [26-Mar-2013 15:58:33] NOTICE: [pool storage] child 20326 exited with code 0 after 0.005202 seconds from start [26-Mar-2013 15:58:33] NOTICE: [pool storage] child 20327 started [26-Mar-2013 15:58:33] NOTICE: [pool storage] child 20327 exited with code 0 after 0.005543 seconds from start [26-Mar-2013 15:58:33] NOTICE: [pool storage] child 20328 started [26-Mar-2013 15:58:33] NOTICE: [pool storage] child 20328 exited with code 0 after 0.004935 seconds from start [26-Mar-2013 15:58:33] NOTICE: [pool storage] child 20329 started [26-Mar-2013 15:58:33] NOTICE: [pool storage] child 20329 exited with code 0 after 0.005255 seconds from start [26-Mar-2013 15:58:33] NOTICE: [pool storage] child 20330 started [26-Mar-2013 15:58:33] NOTICE: [pool storage] child 20330 exited with code 0 after 0.005239 seconds from start [26-Mar-2013 15:58:33] NOTICE: [pool storage] child 20331 started [26-Mar-2013 15:58:33] NOTICE: [pool storage] child 20331 exited with code 0 after 0.005134 seconds from start [26-Mar-2013 15:58:33] NOTICE: [pool storage] child 20332 started [26-Mar-2013 15:58:33] NOTICE: [pool storage] child 20332 exited with code 0 after 0.005162 seconds from start [26-Mar-2013 15:58:33] NOTICE: [pool storage] child 20333 started [26-Mar-2013 15:58:33] NOTICE: [pool storage] child 20333 exited with code 0 after 0.004950 seconds from start [26-Mar-2013 15:58:33] NOTICE: [pool storage] child 20334 started [26-Mar-2013 15:58:33] NOTICE: [pool storage] child 20334 exited with code 0 after 0.005066 seconds from start [26-Mar-2013 15:58:33] NOTICE: [pool storage] child 20335 started ............ ............ ------------------------------------------------------------------------ The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at https://bugs.php.net/bug.php?id=61558 -- Edit this bug report at https://bugs.php.net/bug.php?id=61558&edit=1