Bug #80154 [Opn]: php-fpm segfaults for an unknown reason
Edit report at https://bugs.php.net/bug.php?id=80154&edit=1
ID: 80154
Updated by: nikic@php.net
Reported by: sriccio at swisccenter dot com
Summary: php-fpm segfaults for an unknown reason
Status: Open
Type: Bug
Package: opcache
Operating System: Centos 8
PHP Version: 7.4.10
Block user comment: N
Private report: N
New Comment:
I've been running PHP with file cache against a large number of projects, but didn't
manage to hit this condition. (The one issue I found is PHP 8 specific and fixed in https://github.com/php/php-src/commit/5dfec886d67f01c4e7ea96ef8c26792cb1210047).
@peter dot sopko at backbone dot sk: Would it be possible to run
> p (char*)executor_globals->current_execute_data->func->op_array->filename->val
> p executor_globals->current_execute_data->opline->lineno
> p
> (char*)executor_globals->current_execute_data->func->op_array->scope->name->val
on the core dump, to determine in which PHP file, file and class this occurs? Mainly wondering if
there's anything interesting going on (say, static variables being used inside a trait method).
Previous Comments:
------------------------------------------------------------------------
[2020-11-27 17:14:03] peter dot sopko at backbone dot sk
Hi, it looks like we have (on our shared hosting servers as well) hit the same bug.
We are using FreeBSD (so not CentOS), php is compiled from ports.
settings php_admin_value[opcache.file_cache] = "" seems to mitigate the problem, as well
as offloading the traffic to another php server until the opcache expires (looks like the problem is
in somehow corrupted cached file being created, that is afterwards used when serving requests).
Original opcache config in pool:
php_admin_value[opcache.enable] = 1
php_admin_value[opcache.interned_strings_buffer] = 32
php_admin_value[opcache.max_accelerated_files] = 2000
php_admin_value[opcache.memory_consumption] = 512
php_admin_value[opcache.revalidate_freq] = 900
php_admin_value[opcache.fast_shutdown] = 0
php_admin_value[opcache.enable_cli] = 0
php_admin_value[opcache.revalidate_path] = 0
php_admin_value[opcache.validate_root] = 0
php_admin_value[opcache.validate_permission] = 0
php_admin_value[opcache.validate_timestamps] = 1
php_admin_value[opcache.save_comments] = 1
php_admin_value[opcache.max_file_size] = 0
php_admin_value[opcache.file_cache] = /tmp/
php_admin_value[opcache.file_cache_consistency_checks] = 1
php_admin_value[opcache.file_update_protection] = 2
php_admin_value[opcache.file_cache_only] = 0
php_admin_value[opcache.optimization_level] = 0x7FFFBFFF
Coredump backtrace:
# lldb -c php-fpm.core -- /usr/local/sbin/php-fpm
(lldb) target create "/usr/local/sbin/php-fpm" --core "php-fpm.core"
Core file '/var/tmp/php-fpm.core' (x86_64) was loaded.
(lldb) bt
* thread #1, name = 'php-fpm', stop reason = signal SIGABRT
* frame #0: 0x0000000800d6a1ba libc.so.7`__sys_thr_kill + 10
frame #1: 0x0000000800d685e4 libc.so.7`__raise + 52
frame #2: 0x0000000800cdc7e9 libc.so.7`abort + 73
frame #3: 0x0000000800d58191 libc.so.7`__assert + 81
frame #4: 0x00000000008e1aa5
php-fpm`ZEND_BIND_STATIC_SPEC_CV_UNUSED_HANDLER(execute_data=0x0000000801814680) at
zend_vm_execute.h:46891:3
frame #5: 0x000000000085b014 php-fpm`execute_ex(ex=0x0000000801814020) at
zend_vm_execute.h:53655:7
frame #6: 0x000000000085b24b php-fpm`zend_execute(op_array=0x0000000801872100,
return_value=0x0000000000000000) at zend_vm_execute.h:57957:2
frame #7: 0x00000000007e55a2 php-fpm`zend_execute_scripts(type=8, retval=0x0000000000000000,
file_count=3) at zend.c:1677:4
frame #8: 0x000000000072db47 php-fpm`php_execute_script(primary_file=0x00007fffffffe930) at
main.c:2621:14
frame #9: 0x00000000009121d4 php-fpm`main(argc=1, argv=0x00007fffffffeb80) at fpm_main.c:1942:4
frame #10: 0x00000000003be10f php-fpm`_start(ap=<unavailable>,
cleanup=<unavailable>) at crt1.c:76:7
PHP version used (but the problem happens on previous [7.4.10, 7.4.11] as well)
# php -v
PHP 7.4.12 (cli) (built: Nov 27 2020 15:55:46) ( NTS DEBUG )
Copyright (c) The PHP Group
Zend Engine v3.4.0, Copyright (c) Zend Technologies
with Zend OPcache v7.4.12, Copyright (c), by Zend Technologies
FreeBSD version (but the problem happens on FreeBSD 12.2-RELEASE with latest patches as well);
# uname -a
FreeBSD nas06.backbone.intra 12.1-RELEASE-p10 FreeBSD 12.1-RELEASE-p10 GENERIC amd64
Expected result is of course not to see those coredumps.
------------------------------------------------------------------------
[2020-11-27 09:33:10] nikic@php.net
File cache is plausible here. A fix that looks related based on the backtrace is https://github.com/php/php-src/commit/4609ded082fb5381bb39b408d471e88df9b81674,
but that was already in 7.4.10.
------------------------------------------------------------------------
[2020-11-27 06:27:30] sriccio at swisscenter dot com
Hi,
Just a little followup on this. We still have this issue with PHP 7.4.12 and also latest 7.3.x and
7.2.x.
After some testing it looks like if we disable file_cache_only it stops segfaulting.
So it seems to be related to file_cache ?
------------------------------------------------------------------------
[2020-09-27 08:28:54] sriccio at swisccenter dot com
Description:
------------
On our shared hosting servers, sometime (often) there php-fpm segfaults for an unknown reason.
However we suspect it might be related to opcache, as if we disable it for certain sites, it seems
these aren't segfaulting anymore.
Packages we are using are Remi's php74-php-fpm-7.4.10-1.el8.x86_64$
opcache config in pools
php_admin_value[opcache.file_cache_only] = 1
php_admin_value[opcache.file_cache] = "/home/xyz/.data/opcache/"
php_admin_value[opcache.max_accelerated_files] = 10000
php_admin_value[opcache.memory_consumption] = 128
Expected result:
----------------
No segfault happening
Actual result:
--------------
[root@web21 ~]# coredumpctl info 1817197
PID: 1817197 (php-fpm)
UID: 1098 (xyz)
GID: 1100 (xyz)
Signal: 11 (SEGV)
Timestamp: Sun 2020-09-27 09:56:25 CEST (16min ago)
Command Line: php74-fpm: pool xyz
Executable: /opt/remi/php74/root/usr/sbin/php-fpm
Control Group: /system.slice/php74-php-fpm.service
Unit: php74-php-fpm.service
Slice: system.slice
Boot ID: 504629e6e9164777bb74535d8219c1ee
Machine ID: 06e6f1ebaaae4c8aa0611d132201376a
Hostname: web21
Storage:
/var/lib/systemd/coredump/core.php-fpm.1098.504629e6e9164777bb74535d8219c1ee.1817197.1601193385000000.lz4
Message: Process 1817197 (php-fpm) of user 1098 dumped core.
Stack trace of thread 1817197:
#0 0x000055f36a078540 ZEND_BIND_STATIC_SPEC_CV_UNUSED_HANDLER (php-fpm)
#1 0x000055f36a0afe32 execute_ex (php-fpm)
#2 0x000055f36a0b9661 zend_execute (php-fpm)
#3 0x000055f36a0308fb zend_execute_scripts (php-fpm)
#4 0x000055f369fcdc38 php_execute_script (php-fpm)
#5 0x000055f369e9b850 main (php-fpm)
#6 0x00007fb9677e26a3 __libc_start_main (libc.so.6)
#7 0x000055f369e9c6be _start (php-fpm)
------------------------------------------------------------------------
--
Edit this bug report at https://bugs.php.net/bug.php?id=80154&edit=1
Thread (6 messages)