Bug #61558 [Asn->Csd]: Runaway spawning of children after pipe error

From: Date: Wed, 28 Sep 2022 18:01:43 +0000
Subject: Bug #61558 [Asn->Csd]: Runaway spawning of children after pipe error
References: 1  Groups: php.bugs 
Request: Send a blank email to php-bugs+get-242484@lists.php.net to get a copy of this message
Edit report at https://bugs.php.net/bug.php?id=61558&edit=1

 ID:                 61558
 Updated by:         bukka@php.net
 Reported by:        phpbug2012 at tgabber dot mine dot nu
 Summary:            Runaway spawning of children after pipe error
-Status:             Assigned
+Status:             Closed
 Type:               Bug
 Package:            FPM related
 Operating System:   Debian Linux
 PHP Version:        5.3.10
 Assigned To:        bukka
 Block user comment: N
 Private report:     N



Previous Comments:
------------------------------------------------------------------------
[2022-09-28 18:01:34] bukka@php.net

The actual issue was resolved a long time ago. We kept it open just as a reminder that we should do
something about handling of a quick child crash in FPM master to not become unresponsive. I thought
more about implementation of that and write it down to https://github.com/php/php-src/issues/9632 so
closing this.

------------------------------------------------------------------------
[2015-08-03 03:25:28] kobenews at cox dot net

I think this issue is related to a bug I just posted. Same behavior where php-fpm restarts the
process over and over again, causing php-fpm master to use 100% cpu. The test script is a little bit
simpler, though requires exec() and running a SSH command using multiplexing.

https://bugs.php.net/bug.php?id=70185&edit=2

------------------------------------------------------------------------
[2014-07-25 13:07:22] Danack at basereality dot com

I'd recommend being inspired by aka copying the behaviour of Supervisord.

http://supervisord.org/subprocess.html

Basically it does what it probably the best behaviour of:

i) If child processes exist too quickly, put them in a 'back off' state, to rate limit the
number of processes attempting to be started.

ii) If they still fail to start after a reasonable number of retries stop trying to start it for
now.

Admittedly, sounds like a big task for something that should be a rare event.

------------------------------------------------------------------------
[2014-07-25 09:06:12] tony2001@php.net

Ok, so FPM basically starts the children, but they die immediately (for a reason) and FPM master
process starts the new ones.
I can think of adding some option that would limit the number of children spawned per second, or
probably abort the whole process if N children were created in the last N seconds, but that kind of
solution looks quite hacky to me.

------------------------------------------------------------------------
[2014-07-24 19:44:00] Danack at basereality dot com

I think there is an easier way to trigger this (and probably something that I'll open as a
separate bug):

i) Include a library function in an extension.
ii) Don't include the library in the linking step for php-fpm.
iii) Watch the error log fill up with:


[24-Jul-2014 18:58:27] NOTICE: [pool www] child 22710 started
[24-Jul-2014 18:58:27] WARNING: [pool www] child 22710 exited with code 127 after 0.044963 seconds
from start
[24-Jul-2014 18:58:27] WARNING: [pool www] child 22710 said into stderr: "php-fpm: pool www:
symbol lookup error: /usr/local/lib/php/extensions/no-debug-zts-20131226/imagick.so: undefined
symbol: GetMagickVersion"

Once this happens you need to kill -9 the php-fpm master process to stop it spawning pool workers.

Obviously there shouldn't be errors like this in an extension, but also PHP-FPM shouldn't
become unresponsive and start spawning workers like crazy.

------------------------------------------------------------------------


The remainder of the comments for this report are too long. To view
the rest of the comments, please view the bug report online at

    https://bugs.php.net/bug.php?id=61558


--
Edit this bug report at https://bugs.php.net/bug.php?id=61558&edit=1


Thread (14 messages)

« previous php.bugs (#242484) next »