Req #76225 [Com]: ini setting to not empty opcache when php-fpm is reloaded

Edit report at https://bugs.php.net/bug.php?id=76225&edit=1 ID: 76225 Comment by: post at minhost dot no Reported by: post at minhost dot no Summary: ini setting to not empty opcache when php-fpm is reloaded Status: Open Type: Feature/Change Request Package: opcache Operating System: CentOS 7.4 PHP Version: 7.1.16 Block user comment: N Private report: N New Comment: The crazy thing is that on the test server with a copy of the sites from production server, I only get SIGABRT in php-fpm.log when php-fpm/opcache crashes, but on the production server I get SIGSEGV in php-fpm.log when php-fpm/opcache crashes on the same sites. Previous Comments: ------------------------------------------------------------------------ [2018-04-23 18:52:12] post at minhost dot no Please note that whenever php-fpm/opcache crash, it causes a SIGABRT in php-fpm.log like this: [23-Apr-2018 20:35:50] WARNING: [pool asle] child 949 exited on signal 6 (SIGABRT) after 169.991198 seconds from start And no core dump file is generated in /tmp - is it because it is a SIGABRT and not a SIGSEGV? Should I be able to get a core dump file on crash with SIGABRT? ------------------------------------------------------------------------ [2018-04-23 10:57:22] post at minhost dot no I was able to get one step closer to generating core dumps. It seems I needed to add the following to /etc/security/limits.conf: * soft core unlimited After doing that I was able to get a core file in /tmp when running this command: kill -s SIGSEGV $$ However when php-fpm/opcache crash, there is still not generated any core files in /tmp I have even changed PrivateTmp=true to be PrivateTmp=false - but still no luck. ------------------------------------------------------------------------ [2018-04-20 18:23:02] post at minhost dot no On the test server I have managed to crash two test sites now Some observations I have done now, is that each time I get crash on the test server, it is no more free memory in opcahe, like this (the test server does not have a lot of memory): Used memory 134214488 Free memory 3240 However on the production servers, I have allocated 32 GB to opcache, and I have never seen it use more then 9 GB in opcache. However I am not sure how opcache user the memory limit I set, because the 32 GB is not visible when looking at used memory in shell: [root@production-server ~]# free -m total used free shared buff/cache available Mem: 128658 14825 1961 6672 111870 106292 Swap: 955 954 1 Could it be that opcache is not able to allocate memory from the already used buffer/cache above? Or is the 32 GB memory limit in opcache already accounted for in the current buffer/cache? Anyway, I am still not able to get any dump files in /tmp on the test server. ------------------------------------------------------------------------ [2018-04-20 16:02:40] post at minhost dot no By the way, when I manually double check by looking into the file: /proc/sys/kernel/core_pattern It correctly has this content: /tmp/core-%e.%p And looking into: /proc/sys/kernel/core_uses_pid It correctly has this content: 0 ------------------------------------------------------------------------ [2018-04-20 15:57:37] post at minhost dot no I was finnaly able to get a crash of the copy of a site on a test server. However there was no dump files in /tmp - I really don't know what I did wrong? Here is what I did to activate bactrace: # First I changed all php-fpm.conf config files to include this: rlimit_core = unlimited # Then I compiled PHP-FPM with --enable-debug - and I confirmed a PHP info page says this: "Debug Build yes" # Then I installed GDB with yum: yum install gdb # Then I enabled core dumps in linux like this: su - echo '/tmp/core-%e.%p' > /proc/sys/kernel/core_pattern echo 0 > /proc/sys/kernel/core_uses_pid ulimit -c unlimited # Then i logged out of shell. Then a few days passed without any crash on the test site on the test server. Then today I was able to reproduce a crash on the test server, and PHP-FPM for the site crashed with a 503 error. In apache error log for the test site I got this: [Fri Apr 20 17:30:02.395339 2018] [proxy_fcgi:error] [pid 19834:tid 140118967469824] [client 176.74.214.18:6917] AH01067: Failed to read FastCGI header [Fri Apr 20 17:30:02.395398 2018] [proxy_fcgi:error] [pid 19834:tid 140118967469824] (104)Connection reset by peer: [client 176.74.214.18:6917] AH01075: Error dispatching request to : [Fri Apr 20 17:30:45.711350 2018] [proxy_fcgi:error] [pid 19834:tid 140119471032064] [client 176.74.214.18:6933] AH01067: Failed to read FastCGI header, referer: https://www.fjell-bfk4k.41.no/update.php?op=info [Fri Apr 20 17:30:45.711443 2018] [proxy_fcgi:error] [pid 19834:tid 140119471032064] (104)Connection reset by peer: [client 176.74.214.18:6933] AH01075: Error dispatching request to : , referer: https://www.fjell-bfk4k.41.no/update.php?op=info # Then I went into /tmp to look for core dumps, but ther are not any core dumps in /tmp! By the way I can run gdb just fine like this [root@dns2 ~]# gdb GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-100.el7_4.1 Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>. (gdb) But of course I just get: (gdb) bt No stack. (gdb) help I would like to run: gdb /usr/local/php71/sbin/php-fpm71 /tmp/file-name-to-dump-file.123123 But of course I can't, because there is no dump files in /tmp What am I doing wrong? I really need som help here. ------------------------------------------------------------------------ The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at https://bugs.php.net/bug.php?id=76225 -- Edit this bug report at https://bugs.php.net/bug.php?id=76225&edit=1