Bug #77023 [NEW]: PHP-FPM cannot shutdown processes

From: azurit at pobox dot sk Operating system: Debian 9 PHP version: 7.2.11 Package: FPM related Bug Type: Bug Bug description:PHP-FPM cannot shutdown processes Description: ------------ I'm not albe to reproduce this problem but it is sometimes happening to pool of one of our customers. Pool si set like this: pm = dynamic pm.max_children = 30 pm.start_servers = 1 pm.min_spare_servers = 1 pm.max_spare_servers = 1 I want from it to keep only one IDLE process at a time but FPM sometimes fails to kill IDLE processes (kill because of pm.max_spare_servers) which results into having 30 IDLE processes (or other number, depends on pm.max_children). When i start killing FPM processes by hand, FPM is starting them back BUT there are always few stucked FPM processes (probably the ones which were running for longest time) which, if are killed, FPM itself starts killing also other IDLE processes (like it was waiting for them to shutdown before start shutdowning other processes) and everything goes to normal. I was able to get a strace output from one such process: socket(AF_UNIX, SOCK_STREAM, 0) = 0 rt_sigreturn({mask=[]}) = 43 accept(26, 0x7ffee4d3f860, [112]) = ? ERESTARTSYS (To be restarted if SA_RESTART is set) --- SIGQUIT {si_signo=SIGQUIT, si_code=SI_USER, si_pid=30660, si_uid=0} --- close(0) = 0 socket(AF_UNIX, SOCK_STREAM, 0) = 0 rt_sigreturn({mask=[]}) = 43 accept(26, 0x7ffee4d3f860, [112]) = ? ERESTARTSYS (To be restarted if SA_RESTART is set) --- SIGQUIT {si_signo=SIGQUIT, si_code=SI_USER, si_pid=30660, si_uid=0} --- close(0) = 0 socket(AF_UNIX, SOCK_STREAM, 0) = 0 rt_sigreturn({mask=[]}) = 43 accept(26, 0x7ffee4d3f860, [112]) = ? ERESTARTSYS (To be restarted if SA_RESTART is set) --- SIGQUIT {si_signo=SIGQUIT, si_code=SI_USER, si_pid=30660, si_uid=0} --- close(0) = 0 socket(AF_UNIX, SOCK_STREAM, 0) = 0 rt_sigreturn({mask=[]}) = 43 accept(26, 0x7ffee4d3f860, [112]) = ? ERESTARTSYS (To be restarted if SA_RESTART is set) --- SIGQUIT {si_signo=SIGQUIT, si_code=SI_USER, si_pid=30660, si_uid=0} --- close(0) = 0 socket(AF_UNIX, SOCK_STREAM, 0) = 0 rt_sigreturn({mask=[]}) = 43 accept(26, 0x7ffee4d3f860, [112]) = ? ERESTARTSYS (To be restarted if SA_RESTART is set) --- SIGQUIT {si_signo=SIGQUIT, si_code=SI_USER, si_pid=30660, si_uid=0} --- close(0) = 0 socket(AF_UNIX, SOCK_STREAM, 0) = 0 rt_sigreturn({mask=[]}) = 43 accept(26, 0x7ffee4d3f860, [112]) = ? ERESTARTSYS (To be restarted if SA_RESTART is set) --- SIGQUIT {si_signo=SIGQUIT, si_code=SI_USER, si_pid=30660, si_uid=0} --- close(0) = 0 socket(AF_UNIX, SOCK_STREAM, 0) = 0 rt_sigreturn({mask=[]}) = 43 accept(26, 0x7ffee4d3f860, [112]) = ? ERESTARTSYS (To be restarted if SA_RESTART is set) --- SIGQUIT {si_signo=SIGQUIT, si_code=SI_USER, si_pid=30660, si_uid=0} --- close(0) = 0 socket(AF_UNIX, SOCK_STREAM, 0) = 0 rt_sigreturn({mask=[]}) = 43 accept(26, 0x7ffee4d3f860, [112]) = ? ERESTARTSYS (To be restarted if SA_RESTART is set) --- SIGQUIT {si_signo=SIGQUIT, si_code=SI_USER, si_pid=30660, si_uid=0} --- close(0) = 0 socket(AF_UNIX, SOCK_STREAM, 0) = 0 rt_sigreturn({mask=[]}) = 43 accept(26, 0x7ffee4d3f860, [112]) = ? ERESTARTSYS (To be restarted if SA_RESTART is set) --- SIGQUIT {si_signo=SIGQUIT, si_code=SI_USER, si_pid=30660, si_uid=0} --- close(0) = 0 ... Look on this: # ps aux | grep customer1 | grep fpm | wc -l 11 # ps aux | grep customer1 | grep fpm customer1 2345 0.0 0.1 398280 20640 ? S 12:34 0:00 php-fpm: pool customer1 customer1 3295 0.4 0.2 493180 47964 ? S 12:34 0:12 php-fpm: pool customer1 customer1 10887 0.4 0.2 489452 43628 ? S 12:35 0:12 php-fpm: pool customer1 customer1 12018 0.6 0.3 497272 50852 ? S 12:35 0:18 php-fpm: pool customer1 customer1 12707 0.4 0.3 496552 51748 ? S 12:35 0:14 php-fpm: pool customer1 customer1 13157 0.5 0.4 511312 65676 ? S 12:35 0:15 php-fpm: pool customer1 customer1 15860 0.6 0.3 797176 53464 ? S 12:36 0:17 php-fpm: pool customer1 customer1 15941 0.5 0.3 788804 50856 ? S 12:36 0:16 php-fpm: pool customer1 customer1 23635 0.0 0.1 398280 20636 ? S 13:24 0:00 php-fpm: pool customer1 customer1 23636 0.0 0.1 398280 20636 ? S 13:24 0:00 php-fpm: pool customer1 customer1 23637 0.0 0.1 398280 20636 ? S 13:24 0:00 php-fpm: pool customer1 // the one, which is stucked here, is PID 2345 // let's try to kill few processes except 2345 # skill -kill 10887 12018 13157 23635 23637 # ps aux | grep customer1 | grep fpm | wc -l 11 # ps aux | grep customer1 | grep fpm customer1 2345 0.0 0.1 398280 20640 ? S 12:34 0:00 php-fpm: pool customer1 customer1 3295 0.4 0.3 508332 63120 ? S 12:34 0:14 php-fpm: pool customer1 customer1 12707 0.5 0.3 494504 49876 ? S 12:35 0:15 php-fpm: pool customer1 customer1 15860 0.6 0.4 813560 68608 ? S 12:36 0:19 php-fpm: pool customer1 customer1 15941 0.5 0.3 795164 56780 ? R 12:36 0:17 php-fpm: pool customer1 customer1 23636 0.4 0.2 485492 37128 ? R 13:24 0:00 php-fpm: pool customer1 customer1 26450 2.7 0.2 416048 43144 ? S 13:26 0:00 php-fpm: pool customer1 customer1 26451 2.5 0.2 416048 43144 ? S 13:26 0:00 php-fpm: pool customer1 customer1 26452 2.6 0.2 416048 43156 ? S 13:26 0:00 php-fpm: pool customer1 customer1 26453 2.5 0.2 416048 43140 ? S 13:26 0:00 php-fpm: pool customer1 customer1 26454 0.0 0.1 398660 26028 ? S 13:26 0:00 php-fpm: pool customer1 // now let's kill 2345 # skill -kill 2345 # ps aux | grep customer1 | grep fpm | wc -l 1 # ps aux | grep customer1 | grep fpm customer1 27650 0.1 0.1 398660 26760 ? S 13:27 0:00 php-fpm: pool customer1 -- Edit bug report at https://bugs.php.net/bug.php?id=77023&edit=1 -- Try a snapshot (PHP 5.4): https://bugs.php.net/fix.php?id=77023&r=trysnapshot54 Try a snapshot (PHP 5.5): https://bugs.php.net/fix.php?id=77023&r=trysnapshot55 Try a snapshot (trunk): https://bugs.php.net/fix.php?id=77023&r=trysnapshottrunk Fixed in SVN: https://bugs.php.net/fix.php?id=77023&r=fixed Fixed in release: https://bugs.php.net/fix.php?id=77023&r=alreadyfixed Need backtrace: https://bugs.php.net/fix.php?id=77023&r=needtrace Need Reproduce Script: https://bugs.php.net/fix.php?id=77023&r=needscript Try newer version: https://bugs.php.net/fix.php?id=77023&r=oldversion Not developer issue: https://bugs.php.net/fix.php?id=77023&r=support Expected behavior: https://bugs.php.net/fix.php?id=77023&r=notwrong Not enough info: https://bugs.php.net/fix.php?id=77023&r=notenoughinfo Submitted twice: https://bugs.php.net/fix.php?id=77023&r=submittedtwice register_globals: https://bugs.php.net/fix.php?id=77023&r=globals PHP 4 support discontinued: https://bugs.php.net/fix.php?id=77023&r=php4 Daylight Savings: https://bugs.php.net/fix.php?id=77023&r=dst IIS Stability: https://bugs.php.net/fix.php?id=77023&r=isapi Install GNU Sed: https://bugs.php.net/fix.php?id=77023&r=gnused Floating point limitations: https://bugs.php.net/fix.php?id=77023&r=float No Zend Extensions: https://bugs.php.net/fix.php?id=77023&r=nozend MySQL Configuration Error: https://bugs.php.net/fix.php?id=77023&r=mysqlcfg