Bug #70185 [Opn]: php-fpm restarts master process in a loop when exec() and using ssh multiplexing

Edit report at https://bugs.php.net/bug.php?id=70185&edit=1 ID: 70185 User updated by: kobenews at cox dot net Reported by: kobenews at cox dot net Summary: php-fpm restarts master process in a loop when exec() and using ssh multiplexing Status: Open Type: Bug Package: FPM related Operating System: CentOS release 6.6 (Final) PHP Version: 5.4.43 Block user comment: N Private report: N New Comment: php at plainview dot se: glad I'm not the only one experiencing this bug. Now the trick, is to get a fix. I am running: PHP 5.4.43 on CentOS 6.7 Previous Comments: ------------------------------------------------------------------------ [2015-09-08 20:39:28] php at plainview dot se Finally, someone else with the same bug! I have been dealing with it for years and, until I read this, used a separate php-fastcgi to deal with the constant child spamming. My previous setup was a Debian 7 with 5.4.4-14+deb7u14. My current setup is a Debian 8 with 5.6.9+dfsg-0+deb8u1. Both exhibit the exact same behaviour: when a multiplexed SSH is exec()ed it goes spawn crazy, with children dying left and right: [08-Sep-2015 21:32:36] NOTICE: [pool www] child 30721 exited with code 0 after 0.011428 seconds from start [08-Sep-2015 21:32:36] NOTICE: [pool www] child 30722 exited with code 0 after 0.011916 seconds from start [08-Sep-2015 21:32:36] NOTICE: [pool www] child 30723 exited with code 0 after 0.011527 seconds from start [08-Sep-2015 21:32:36] NOTICE: [pool www] child 30724 exited with code 0 after 0.012399 seconds from start [08-Sep-2015 21:32:36] NOTICE: [pool www] child 30725 exited with code 0 after 0.011333 seconds from start [08-Sep-2015 21:32:36] NOTICE: [pool www] child 30726 exited with code 0 after 0.010918 seconds from start [08-Sep-2015 21:32:36] NOTICE: [pool www] child 30727 exited with code 0 after 0.010977 seconds from start [08-Sep-2015 21:32:36] NOTICE: [pool www] child 30728 exited with code 0 after 0.011124 seconds from start The result is 100% CPU load and a sluggish system. The source of the problem is, as kobenews says, multiplexing SSH. I do it via a series of bash scripts that are execed by PHP. Original script: #!/bin/bash CONTROLPATH=/var/www/mail_scripts/master-mail_scripts MAIL_SCRIPTS_COMMAND="ssh -o StrictHostKeyChecking=no -l mail_scripts -o ControlPath=$CONTROLPATH -i /var/www/mail_scripts/mail_scripts TARGET.COM" # If the socket doesn't exist, create it. if [ ! -e $CONTROLPATH* ] ; then $MAIL_SCRIPTS_COMMAND -fNM fi # Execute the requested command. $MAIL_SCRIPTS_COMMAND /home/mail_scripts/the_command $@ What I did was I moveed -f in front of TARGET.COM and started the master manually, outside of PHP. After that PHP seems fine with running the script without spawnflooding. ------------------------------------------------------------------------ [2015-08-07 20:13:58] kobenews at cox dot net Updated summary. ------------------------------------------------------------------------ [2015-08-03 03:21:06] kobenews at cox dot net Sorry, the script I provided is incorrect. I forgot a command at the end. It should be: <?php $command = 'ssh -o "ControlMaster=auto" -o "ControlPersist=2m" -S /home/justin/.ssh/sockets/justin.sock justin@1.2.3.4 uptime'; $output = exec($command, $output_array, $return_code); die($output); ------------------------------------------------------------------------ [2015-08-03 02:54:44] kobenews at cox dot net Description: ------------ Ok bear with me, this bug is a bit obscure and takes some setup. However, it is consistent, and causes php-fpm to continue to restart the process over and over again using 100% of the CPU on the php-fpm master process. It only occurs with SSH controlMaster which basically allows SSH connection multiplexing. Here is an example code: <?php $command = 'ssh -o "ControlMaster=auto" -o "ControlPersist=2m" -S /home/justin/.ssh/sockets/justin.sock justin@1.2.3.4'; $output = exec($command, $output_array, $return_code); If you remove the options -o "ControlMaster=auto" -o "ControlPersist=2m" -S /home/justin/.ssh/sockets/justin.sock this works without crashing over and over again. Test script: --------------- <?php $command = 'ssh -o "ControlMaster=auto" -o "ControlPersist=2m" -S /home/justin/.ssh/sockets/justin.sock justin@1.2.3.4'; $output = exec($command, $output_array, $return_code); Expected result: ---------------- php-fpm master process should not restart the process over and over again. Nothing is crashing here. Actual result: -------------- php-fpm continues to restart the process over and over again using 100% of the CPU on the php-fpm master process. ------------------------------------------------------------------------ -- Edit this bug report at https://bugs.php.net/bug.php?id=70185&edit=1