Edit report at https://bugs.php.net/bug.php?id=70185&edit=1
ID: 70185
Comment by: php at plainview dot se
Reported by: kobenews at cox dot net
Summary: php-fpm restarts master process in a loop when
exec() and using ssh multiplexing
Status: Open
Type: Bug
Package: FPM related
Operating System: CentOS release 6.6 (Final)
PHP Version: 5.4.43
Block user comment: N
Private report: N
New Comment:
Finally, someone else with the same bug! I have been dealing with it for years and, until I read
this, used a separate php-fastcgi to deal with the constant child spamming.
My previous setup was a Debian 7 with 5.4.4-14+deb7u14.
My current setup is a Debian 8 with 5.6.9+dfsg-0+deb8u1.
Both exhibit the exact same behaviour: when a multiplexed SSH is exec()ed it goes spawn crazy, with
children dying left and right:
[08-Sep-2015 21:32:36] NOTICE: [pool www] child 30721 exited with code 0 after 0.011428 seconds from
start
[08-Sep-2015 21:32:36] NOTICE: [pool www] child 30722 exited with code 0 after 0.011916 seconds from
start
[08-Sep-2015 21:32:36] NOTICE: [pool www] child 30723 exited with code 0 after 0.011527 seconds from
start
[08-Sep-2015 21:32:36] NOTICE: [pool www] child 30724 exited with code 0 after 0.012399 seconds from
start
[08-Sep-2015 21:32:36] NOTICE: [pool www] child 30725 exited with code 0 after 0.011333 seconds from
start
[08-Sep-2015 21:32:36] NOTICE: [pool www] child 30726 exited with code 0 after 0.010918 seconds from
start
[08-Sep-2015 21:32:36] NOTICE: [pool www] child 30727 exited with code 0 after 0.010977 seconds from
start
[08-Sep-2015 21:32:36] NOTICE: [pool www] child 30728 exited with code 0 after 0.011124 seconds from
start
The result is 100% CPU load and a sluggish system.
The source of the problem is, as kobenews says, multiplexing SSH. I do it via a series of bash
scripts that are execed by PHP.
Original script:
#!/bin/bash
CONTROLPATH=/var/www/mail_scripts/master-mail_scripts
MAIL_SCRIPTS_COMMAND="ssh -o StrictHostKeyChecking=no -l mail_scripts -o
ControlPath=$CONTROLPATH -i /var/www/mail_scripts/mail_scripts TARGET.COM"
# If the socket doesn't exist, create it.
if [ ! -e $CONTROLPATH* ] ; then
$MAIL_SCRIPTS_COMMAND -fNM
fi
# Execute the requested command.
$MAIL_SCRIPTS_COMMAND /home/mail_scripts/the_command $@
What I did was I moveed -f in front of TARGET.COM and started the master manually, outside of PHP.
After that PHP seems fine with running the script without spawnflooding.
Previous Comments:
------------------------------------------------------------------------
[2015-08-07 20:13:58] kobenews at cox dot net
Updated summary.
------------------------------------------------------------------------
[2015-08-03 03:21:06] kobenews at cox dot net
Sorry, the script I provided is incorrect. I forgot a command at the end. It should be:
<?php
$command = 'ssh -o "ControlMaster=auto" -o "ControlPersist=2m" -S
/home/justin/.ssh/sockets/justin.sock justin@1.2.3.4 uptime';
$output = exec($command, $output_array, $return_code);
die($output);
------------------------------------------------------------------------
[2015-08-03 02:54:44] kobenews at cox dot net
Description:
------------
Ok bear with me, this bug is a bit obscure and takes some setup. However, it is consistent, and
causes php-fpm to continue to restart the process over and over again using 100% of the CPU on the
php-fpm master process.
It only occurs with SSH controlMaster which basically allows SSH connection multiplexing.
Here is an example code:
<?php
$command = 'ssh -o "ControlMaster=auto" -o "ControlPersist=2m" -S
/home/justin/.ssh/sockets/justin.sock justin@1.2.3.4';
$output = exec($command, $output_array, $return_code);
If you remove the options -o "ControlMaster=auto" -o "ControlPersist=2m" -S
/home/justin/.ssh/sockets/justin.sock this works without crashing over and over again.
Test script:
---------------
<?php
$command = 'ssh -o "ControlMaster=auto" -o "ControlPersist=2m" -S
/home/justin/.ssh/sockets/justin.sock justin@1.2.3.4';
$output = exec($command, $output_array, $return_code);
Expected result:
----------------
php-fpm master process should not restart the process over and over again. Nothing is crashing here.
Actual result:
--------------
php-fpm continues to restart the process over and over again using 100% of the CPU on the php-fpm
master process.
------------------------------------------------------------------------
--
Edit this bug report at https://bugs.php.net/bug.php?id=70185&edit=1