Bug #77653 [Asn]: php-fpm, operator displayed instead of the real error message

Edit report at https://bugs.php.net/bug.php?id=77653&edit=1 ID: 77653 Updated by: bukka@php.net Reported by: claudiu_beta at yahoo dot com Summary: php-fpm, operator displayed instead of the real error message Status: Assigned Type: Bug Package: FPM related Operating System: Fedora 29 PHP Version: 7.3.3RC1 Assigned To: bukka Block user comment: N Private report: N New Comment: I'm afraid I don't have any changes that would fix your issue as I don't know what the real issue is. I need to be able to recreate it. Basically I will need a minimal app that shows the issue. Ideally something like the following that I used for another issue: https://github.com/bukka/php-util/tree/master/tests/fpm/pools-reload Of course the configuration should be different and mainly the input script should behave in a way that I can see the issue. If you are able to provide something like that, I should be able to fix it hopefully but otherwise I can't really do much. I will be quite busy in the next 3 weeks but then should be able to take a look if you manage to extract it. Previous Comments: ------------------------------------------------------------------------ [2019-03-03 23:36:05] claudiu_beta at yahoo dot com I don't think it has something to do with killed children, because the issue is happening with the same sites (1 site = 1 pool). Always the same error output (child xxxx said into stderr: "^@") and never a normal error message for these pools. As for the 7.2 log, I don't see empty error messages "", all are filled with data. Also, I have checked few pools and they do not ever have entries in slow.log. Would be great if you can include those changes with the next 7.3.3 release, so I can test and see if bug was solved or not. Sadly, I don't compile php from sources and I can't test before the public release. I only use rpm packages for Fedora 29. And on a local server I can't reproduce the error myself, so any test is useless. I'm running php 7.3 on few servers. From 500+ pools, on each server, I have the same 3-4-5 pools always displaying the error output in binary format and few others displaying errors without problem. Sites are not connected each other. I have checked the sites to see a link between them: one is Chinese, one is English US, one Spanish etc. I thought it may be a foreign language, but it's happening with sites in English too. I have checked few pools for errors in master log and also the individual log set in pool conf with php_admin_value[error_log] = xxxxx.log [pool example] I have in master log [01-Mar-2019 21:08:30] WARNING: [pool example2] child 21835 said into stderr: "^@" [01-Mar-2019 21:08:30] WARNING: [pool example2] child 21835 said into stderr: "^@" but nothing recent in example.log [21-Feb-2019 18:33:50 Europe/Paris] PHP Parse error: syntax error, unexpected ';' in /path/to/wstsczona.php on line 15 [21-Feb-2019 18:34:45 Europe/Paris] PHP Parse error: syntax error, unexpected '$conexion' (T_VARIABLE) in /path/to/wstsczona.php on line 17 so php code errors and php-fpm warnings seems to not be related. ------------------------------------------------------------------------ [2019-03-03 17:18:08] bukka@php.net Ok so it happens rarely just for some pools, right? I'm wondering if it could be somehow related to the killed children. Maybe something related to request_slowlog_timeout and request_terminate_timeout? Could you also get result from the scoreboard and compare it with other working pools. And looking to www-slow.log could help too. One of the things that changed in 7.3 is that binary content logging is supported better and technically strings starting with '\0' could still be logged (not really tested but I replaced strchr with memchr and changed the logic a bit so it might work). It would be interesting to know if the binary content starts with '\0' and if you had some empty log entries in 7.2 (...said into stderr: "")? ------------------------------------------------------------------------ [2019-02-24 16:56:50] claudiu_beta at yahoo dot com Please note that I'm running ~500 php7.3 pools with the same pool configuration, except the path. This is a very rare incident, because, as you can see, starting with Feb 22, from 500+ pools only 2 display the error wrongly and one displays the error correctly. ------------------------------------------------------------------------ [2019-02-24 16:47:36] claudiu_beta at yahoo dot com error_reporting = E_ALL & ~E_NOTICE & ~E_DEPRECATED ------------------------------------------------------------------------ [2019-02-24 16:38:00] claudiu_beta at yahoo dot com I'm running multiple php versions using software collection from Remi repo. Similar configuration for all versions. I only have issues with 7.3. Installed packages ------------------------- php73-php-debuginfo-7.3.1~RC1-1.fc28.remi.x86_64 php73-php-mbstring-7.3.3~RC1-1.fc29.remi.x86_64 php73-runtime-2.0-1.fc29.remi.x86_64 php73-2.0-1.fc29.remi.x86_64 php73-php-gmp-7.3.3~RC1-1.fc29.remi.x86_64 php73-php-xmlrpc-7.3.3~RC1-1.fc29.remi.x86_64 php73-build-2.0-1.fc29.remi.x86_64 php73-php-mysqlnd-7.3.3~RC1-1.fc29.remi.x86_64 php73-php-fpm-debuginfo-7.3.1~RC1-1.fc28.remi.x86_64 php73-php-process-7.3.3~RC1-1.fc29.remi.x86_64 php73-php-bcmath-7.3.3~RC1-1.fc29.remi.x86_64 php73-php-soap-7.3.3~RC1-1.fc29.remi.x86_64 php73-php-pecl-msgpack-2.0.3-1.fc29.remi.x86_64 php73-php-pecl-imagick-3.4.3-13.fc29.remi.x86_64 php73-php-pecl-igbinary-3.0.0-1.fc29.remi.x86_64 php73-php-common-7.3.3~RC1-1.fc29.remi.x86_64 php73-php-intl-7.3.3~RC1-1.fc29.remi.x86_64 php73-php-pear-1.10.8-1.fc29.remi.noarch php73-php-ioncube-loader-10.3.2-1.fc29.remi.x86_64 php73-php-pecl-zip-1.15.4-1.fc29.remi.x86_64 php73-php-cli-7.3.3~RC1-1.fc29.remi.x86_64 php73-php-json-7.3.3~RC1-1.fc29.remi.x86_64 php73-php-imap-7.3.3~RC1-1.fc29.remi.x86_64 php73-php-pdo-7.3.3~RC1-1.fc29.remi.x86_64 php73-php-fpm-7.3.3~RC1-1.fc29.remi.x86_64 php73-php-pecl-memcached-3.1.3-1.fc29.remi.x86_64 php73-php-xml-7.3.3~RC1-1.fc29.remi.x86_64 php73-php-devel-7.3.3~RC1-1.fc29.remi.x86_64 php73-php-gd-7.3.3~RC1-1.fc29.remi.x86_64 php-fpm.conf ------------------ include=/etc/opt/remi/php73/php-fpm.d/*.conf [global] pid = /var/opt/remi/php73/run/php-fpm/php-fpm.pid error_log = /var/log/php-fpm/master73.log syslog.ident = php-fpm73 daemonize = yes rlimit_files = 30000 ------------------------------------ And a pool. ------------ [poolname] user = apache group = apache listen = /var/opt/remi/php73/run/php-fpm/pool.sock listen.acl_users = apache listen.allowed_clients = 127.0.0.1 pm = ondemand pm.max_children = 20 pm.process_idle_timeout = 5s pm.max_requests = 40 pm.status_path = /fpmstats slowlog = /var/log/php-fpm/www-slow.log request_slowlog_timeout = 200 request_terminate_timeout = 203 chdir = /home/user catch_workers_output = yes security.limit_extensions = .php .phtml .phar php_admin_value[doc_root] = "/home/user" ============================================ For grep I must add -a, because that ^Q visible in console is binary content. grep -a stderr /var/log/php-fpm/master73.log ------------------ [22-Feb-2019 04:56:01] WARNING: [pool poolA] child 9662 said into stderr: "^@" [22-Feb-2019 04:57:01] WARNING: [pool poolA] child 9963 said into stderr: "^@" [22-Feb-2019 04:57:59] WARNING: [pool poolA] child 10262 said into stderr: "^@" [22-Feb-2019 05:01:02] WARNING: [pool poolA] child 11205 said into stderr: "^@" [22-Feb-2019 05:04:01] WARNING: [pool poolA] child 12168 said into stderr: "^@" [22-Feb-2019 05:17:00] WARNING: [pool poolA] child 17024 said into stderr: "^@" [22-Feb-2019 05:18:02] WARNING: [pool poolA] child 17347 said into stderr: "^@" [22-Feb-2019 05:23:00] WARNING: [pool poolA] child 19405 said into stderr: "^@" [22-Feb-2019 05:28:01] WARNING: [pool poolA] child 21458 said into stderr: "^@" [22-Feb-2019 05:46:01] WARNING: [pool poolA] child 28293 said into stderr: "^@" [22-Feb-2019 05:48:00] WARNING: [pool poolA] child 29068 said into stderr: "^@" [22-Feb-2019 05:50:59] WARNING: [pool poolA] child 30215 said into stderr: "^@" [22-Feb-2019 05:53:02] WARNING: [pool poolA] child 31052 said into stderr: "^@" [21-Feb-2019 14:19:58] WARNING: [pool poolB] child 9011 said into stderr: "ERROR: Unable to open primary script: /home/user/index.php (Permission denied)" [21-Feb-2019 15:23:34] WARNING: [pool poolC] child 4159 said into stderr: "^@" [21-Feb-2019 15:25:52] WARNING: [pool poolC] child 5151 said into stderr: "^@" [22-Feb-2019 03:34:08] WARNING: [pool poolC] child 11060 said into stderr: "^@" [22-Feb-2019 03:56:00] WARNING: [pool poolA] child 19510 said into stderr: "^@" [22-Feb-2019 04:16:01] WARNING: [pool poolA] child 26999 said into stderr: "^@" [22-Feb-2019 04:18:00] WARNING: [pool poolA] child 27707 said into stderr: "^@" [22-Feb-2019 04:30:00] WARNING: [pool poolA] child 32308 said into stderr: "^@" [22-Feb-2019 04:34:01] WARNING: [pool poolA] child 1579 said into stderr: "^@" [22-Feb-2019 04:46:02] WARNING: [pool poolA] child 6239 said into stderr: "^@" [22-Feb-2019 04:51:01] WARNING: [pool poolA] child 8030 said into stderr: "^@" [22-Feb-2019 04:56:01] WARNING: [pool poolA] child 9662 said into stderr: "^@" [22-Feb-2019 04:57:01] WARNING: [pool poolA] child 9963 said into stderr: "^@" [22-Feb-2019 04:57:59] WARNING: [pool poolA] child 10262 said into stderr: "^@" [22-Feb-2019 05:01:02] WARNING: [pool poolA] child 11205 said into stderr: "^@" [22-Feb-2019 05:04:01] WARNING: [pool poolA] child 12168 said into stderr: "^@" [22-Feb-2019 05:17:00] WARNING: [pool poolA] child 17024 said into stderr: "^@" [22-Feb-2019 05:18:02] WARNING: [pool poolA] child 17347 said into stderr: "^@" [22-Feb-2019 05:23:00] WARNING: [pool poolA] child 19405 said into stderr: "^@" [22-Feb-2019 05:28:01] WARNING: [pool poolA] child 21458 said into stderr: "^@" [22-Feb-2019 05:46:01] WARNING: [pool poolA] child 28293 said into stderr: "^@" [22-Feb-2019 05:48:00] WARNING: [pool poolA] child 29068 said into stderr: "^@" [22-Feb-2019 05:50:59] WARNING: [pool poolA] child 30215 said into stderr: "^@" [22-Feb-2019 05:53:02] WARNING: [pool poolA] child 31052 said into stderr: "^@" [22-Feb-2019 05:56:02] WARNING: [pool poolA] child 31911 said into stderr: "^@" [22-Feb-2019 05:58:01] WARNING: [pool poolA] child 32449 said into stderr: "^@" [22-Feb-2019 06:02:59] WARNING: [pool poolA] child 1854 said into stderr: "^@" [22-Feb-2019 06:05:01] WARNING: [pool poolA] child 2804 said into stderr: "^@" [22-Feb-2019 06:15:01] WARNING: [pool poolA] child 6377 said into stderr: "^@" [22-Feb-2019 06:20:02] WARNING: [pool poolA] child 8086 said into stderr: "^@" [22-Feb-2019 06:23:01] WARNING: [pool poolA] child 9356 said into stderr: "^@" [22-Feb-2019 06:44:02] WARNING: [pool poolA] child 17610 said into stderr: "^@" [22-Feb-2019 06:50:02] WARNING: [pool poolA] child 19670 said into stderr: "^@" [22-Feb-2019 06:55:01] WARNING: [pool poolA] child 21604 said into stderr: "^@" [22-Feb-2019 07:00:01] WARNING: [pool poolA] child 23429 said into stderr: "^@" [22-Feb-2019 07:04:01] WARNING: [pool poolA] child 25114 said into stderr: "^@" [22-Feb-2019 07:06:01] WARNING: [pool poolA] child 25821 said into stderr: "^@" [22-Feb-2019 07:07:01] WARNING: [pool poolA] child 26061 said into stderr: "^@" [22-Feb-2019 07:08:00] WARNING: [pool poolA] child 26293 said into stderr: "^@" [22-Feb-2019 07:13:02] WARNING: [pool poolA] child 27602 said into stderr: "^@" [22-Feb-2019 07:23:02] WARNING: [pool poolA] child 30443 said into stderr: "^@" [22-Feb-2019 07:24:03] WARNING: [pool poolA] child 30788 said into stderr: "^@" [22-Feb-2019 07:26:02] WARNING: [pool poolA] child 31286 said into stderr: "^@" [22-Feb-2019 07:34:00] WARNING: [pool poolA] child 1427 said into stderr: "^@" [22-Feb-2019 07:35:00] WARNING: [pool poolA] child 1686 said into stderr: "^@" [22-Feb-2019 07:36:02] WARNING: [pool poolA] child 2044 said into stderr: "^@" [22-Feb-2019 07:43:02] WARNING: [pool poolA] child 3962 said into stderr: "^@" [22-Feb-2019 07:44:01] WARNING: [pool poolA] child 4216 said into stderr: "^@" [22-Feb-2019 07:54:01] WARNING: [pool poolA] child 7335 said into stderr: "^@" [22-Feb-2019 07:57:01] WARNING: [pool poolA] child 8178 said into stderr: "^@" [22-Feb-2019 08:05:01] WARNING: [pool poolA] child 10447 said into stderr: "^@" [22-Feb-2019 08:06:01] WARNING: [pool poolA] child 10816 said into stderr: "^@" [22-Feb-2019 08:09:01] WARNING: [pool poolA] child 11720 said into stderr: "^@" [22-Feb-2019 08:11:05] WARNING: [pool poolA] child 12570 said into stderr: "^@" [22-Feb-2019 08:17:00] WARNING: [pool poolA] child 14056 said into stderr: "^@" [22-Feb-2019 08:25:00] WARNING: [pool poolA] child 16595 said into stderr: "^@" [22-Feb-2019 08:28:01] WARNING: [pool poolA] child 17291 said into stderr: "^@" [22-Feb-2019 08:29:00] WARNING: [pool poolA] child 17527 said into stderr: "^@" [22-Feb-2019 08:49:01] WARNING: [pool poolA] child 22856 said into stderr: "^@" [22-Feb-2019 08:52:02] WARNING: [pool poolA] child 23818 said into stderr: "^@" [22-Feb-2019 08:54:01] WARNING: [pool poolA] child 24670 said into stderr: "^@" [22-Feb-2019 09:00:01] WARNING: [pool poolA] child 26495 said into stderr: "^@" [22-Feb-2019 09:03:00] WARNING: [pool poolA] child 27733 said into stderr: "^@" [23-Feb-2019 14:34:34] WARNING: [pool poolB] child 32366 said into stderr: "ERROR: Unable to open primary script: /home/user/index.php (Permission denied)" [24-Feb-2019 01:17:10] WARNING: [pool poolB] child 13089 said into stderr: "ERROR: Unable to open primary script: /home/user/index.php (Permission denied)" [24-Feb-2019 08:27:13] WARNING: [pool poolB] child 9651 said into stderr: "ERROR: Unable to open primary script: /home/user/index.php (Permission denied)" [24-Feb-2019 17:53:02] WARNING: [pool poolB] child 30341 said into stderr: "ERROR: Unable to open primary script: /home/user/index.php (Permission denied)" ================================= Probably my "working" example including "Unable to open primary script:" was not a good one, because in this case error seems to be displayed correctly. Other errors are not displayed. ------------------------------------------------------------------------ The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at https://bugs.php.net/bug.php?id=77653 -- Edit this bug report at https://bugs.php.net/bug.php?id=77653&edit=1