Bug #74403 [Asn]: Sporadic FPM segfaults

Edit report at https://bugs.php.net/bug.php?id=74403&edit=1 ID: 74403 Updated by: laruence@php.net Reported by: tony2001@php.net Summary: Sporadic FPM segfaults Status: Assigned Type: Bug Package: PCRE related Operating System: Linux PHP Version: 7.1.4RC1 Assigned To: laruence Block user comment: N Private report: N New Comment: from the backtrace, it's really not like a problem that is the "key change" caused could you try without memtrack ? anyway, @tony2001, I understand your concern, but we should try to find the root cause, you patch may wroks, but it also may cover some real problems. Previous Comments: ------------------------------------------------------------------------ [2017-04-11 09:51:42] tony2001@php.net Reverted the attached patch in my private branch and deployed FPM on one production server. After about 6 hours I've got several cores with the backtraces below. Number of segfaults on several hundred servers since March 22: 0. backtrace #1: #0 zend_hash_find_bucket (ht=ht@entry=0xdd28e1cadaf9342c, ht=ht@entry=0xdd28e1cadaf9342c, key=0x7f37b66bdf00, key@entry=0x7f57b453b718) at /home/tony/php-build/php-src/Zend/zend_hash.c:496 #1 zend_hash_find (ht=ht@entry=0x7f37b4539720, key=key@entry=0x7f37b66bdf00) at /home/tony/php-build/php-src/Zend/zend_hash.c:1988 #2 0x00000000006da2e6 in zend_hash_find_ind (key=0x7f37b66bdf00, ht=0x7f37b4539720) at /home/tony/php-build/php-src/Zend/zend_hash.h:278 #3 ZEND_ISSET_ISEMPTY_DIM_OBJ_SPEC_CV_CV_HANDLER () at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:37856 #4 0x00000000006b0e4b in execute_ex (ex=<optimized out>) at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:414 #5 0x00007f382e4ca31f in memtrack_execute_ex (execute_data=0x7f3833c18720) at /home/tony/php-build/php-src/ext/memtrack/memtrack.c:481 #6 0x00000000006f115d in ZEND_DO_FCALL_SPEC_HANDLER () at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:800 #7 0x00000000006b0e4b in execute_ex (ex=<optimized out>) at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:414 #8 0x00007f382e4ca31f in memtrack_execute_ex (execute_data=0x7f3833c18660) at /home/tony/php-build/php-src/ext/memtrack/memtrack.c:481 #9 0x00000000006f115d in ZEND_DO_FCALL_SPEC_HANDLER () at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:800 #10 0x00000000006b0e4b in execute_ex (ex=<optimized out>) at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:414 #11 0x00007f382e4ca31f in memtrack_execute_ex (execute_data=0x7f3833c18480) at /home/tony/php-build/php-src/ext/memtrack/memtrack.c:481 #12 0x000000000065b919 in zend_call_function (fci=fci@entry=0x7ffcffa5d8d0, fci_cache=fci_cache@entry=0x7ffcffa5d8a0) at /home/tony/php-build/php-src/Zend/zend_execute_API.c:858 #13 0x0000000000689db3 in zend_call_method (object=0x0, obj_ce=<optimized out>, fn_proxy=0x7f3833c55870, function_name=0x7f3833c74248 "main_autoload", function_name_len=<optimized out>, retval_ptr=retval_ptr@entry=0x0, param_count=1, arg1=0x7f3833c18470, arg2=0x0) at /home/tony/php-build/php-src/Zend/zend_interfaces.c:104 #14 0x0000000000525c4e in zif_spl_autoload_call (execute_data=<optimized out>, return_value=<optimized out>) at /home/tony/php-build/php-src/ext/spl/php_spl.c:421 #15 0x00007f382e4ca680 in memtrack_execute_internal (current_execute_data=0x7f3833c18410, return_value=0x7ffcffa5dbc0) at /home/tony/php-build/php-src/ext/memtrack/memtrack.c:563 #16 0x000000000065bb57 in zend_call_function (fci=fci@entry=0x7ffcffa5dc00, fci_cache=fci_cache@entry=0x7ffcffa5dbd0) at /home/tony/php-build/php-src/Zend/zend_execute_API.c:880 #17 0x000000000065bee2 in zend_lookup_class_ex (name=name@entry=0x7f37b66bdf00, key=key@entry=0x0, use_autoload=use_autoload@entry=1) at /home/tony/php-build/php-src/Zend/zend_execute_API.c:1040 #18 0x000000000065c635 in zend_fetch_class (class_name=0x7f37b66bdf00, fetch_type=512) at /home/tony/php-build/php-src/Zend/zend_execute_API.c:1364 #19 0x00000000006e3d2a in ZEND_FETCH_CLASS_SPEC_CV_HANDLER () at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:2330 #20 0x00000000006b0e4b in execute_ex (ex=<optimized out>) at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:414 #21 0x00007f382e4ca31f in memtrack_execute_ex (execute_data=0x7f3833c18070) at /home/tony/php-build/php-src/ext/memtrack/memtrack.c:481 #22 0x00000000006f115d in ZEND_DO_FCALL_SPEC_HANDLER () at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:800 #23 0x00000000006b0e4b in execute_ex (ex=<optimized out>) at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:414 #24 0x00007f382e4ca31f in memtrack_execute_ex (execute_data=0x7f3833c14e60) at /home/tony/php-build/php-src/ext/memtrack/memtrack.c:481 #25 0x00000000006f115d in ZEND_DO_FCALL_SPEC_HANDLER () at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:800 #26 0x00000000006b0e4b in execute_ex (ex=<optimized out>) at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:414 #27 0x00007f382e4ca31f in memtrack_execute_ex (execute_data=0x7f3833c134c0) at /home/tony/php-build/php-src/ext/memtrack/memtrack.c:481 #28 0x00000000006f115d in ZEND_DO_FCALL_SPEC_HANDLER () at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:800 (gdb) p p $1 = (Bucket *) 0x7f57b453b718 (gdb) p *p Cannot access memory at address 0x7f57b453b718 backtrace #2: #0 zend_verify_abstract_class (ce=0x7f37a95d414000) at /home/tony/php-build/php-src/Zend/zend_execute_API.c:1441 #1 0x00000000006b6f24 in ZEND_VERIFY_ABSTRACT_CLASS_SPEC_HANDLER () at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:1539 #2 0x00000000006b0e4b in execute_ex (ex=<optimized out>) at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:414 #3 0x00007f382e4ca31f in memtrack_execute_ex (execute_data=0x7f3833c15af0) at /home/tony/php-build/php-src/ext/memtrack/memtrack.c:481 #4 0x000000000071673c in ZEND_INCLUDE_OR_EVAL_SPEC_CV_HANDLER () at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:29532 #5 0x00000000006b0e4b in execute_ex (ex=<optimized out>) at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:414 #6 0x00007f382e4ca31f in memtrack_execute_ex (execute_data=0x7f3833c15720) at /home/tony/php-build/php-src/ext/memtrack/memtrack.c:481 #7 0x00000000006f115d in ZEND_DO_FCALL_SPEC_HANDLER () at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:800 #8 0x00000000006b0e4b in execute_ex (ex=<optimized out>) at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:414 #9 0x00007f382e4ca31f in memtrack_execute_ex (execute_data=0x7f3833c156a0) at /home/tony/php-build/php-src/ext/memtrack/memtrack.c:481 #10 0x00000000006f115d in ZEND_DO_FCALL_SPEC_HANDLER () at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:800 #11 0x00000000006b0e4b in execute_ex (ex=<optimized out>) at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:414 #12 0x00007f382e4ca31f in memtrack_execute_ex (execute_data=0x7f3833c154c0) at /home/tony/php-build/php-src/ext/memtrack/memtrack.c:481 #13 0x000000000065b919 in zend_call_function (fci=fci@entry=0x7ffcffa5d0c0, fci_cache=fci_cache@entry=0x7ffcffa5d090) at /home/tony/php-build/php-src/Zend/zend_execute_API.c:858 #14 0x0000000000689db3 in zend_call_method (object=0x0, obj_ce=<optimized out>, fn_proxy=0x7f3833c55810, function_name=0x7f3833c762e8 "main_autoload", function_name_len=<optimized out>, retval_ptr=retval_ptr@entry=0x0, param_count=1, arg1=0x7f3833c154b0, arg2=0x0) at /home/tony/php-build/php-src/Zend/zend_interfaces.c:104 #15 0x0000000000525c4e in zif_spl_autoload_call (execute_data=<optimized out>, return_value=<optimized out>) at /home/tony/php-build/php-src/ext/spl/php_spl.c:421 #16 0x00007f382e4ca680 in memtrack_execute_internal (current_execute_data=0x7f3833c15450, return_value=0x7ffcffa5d3b0) at /home/tony/php-build/php-src/ext/memtrack/memtrack.c:563 #17 0x000000000065bb57 in zend_call_function (fci=fci@entry=0x7ffcffa5d3f0, fci_cache=fci_cache@entry=0x7ffcffa5d3c0) at /home/tony/php-build/php-src/Zend/zend_execute_API.c:880 #18 0x000000000065bee2 in zend_lookup_class_ex (name=name@entry=0x7f37b4487cd0, key=0x7f37b4487b98, use_autoload=use_autoload@entry=1) at /home/tony/php-build/php-src/Zend/zend_execute_API.c:1040 #19 0x000000000065c7d8 in zend_fetch_class_by_name (class_name=0x7f37b4487cd0, key=<optimized out>, fetch_type=fetch_type@entry=512) at /home/tony/php-build/php-src/Zend/zend_execute_API.c:1386 #20 0x00000000007148d7 in ZEND_INIT_STATIC_METHOD_CALL_SPEC_CONST_CONST_HANDLER () at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:5667 #21 0x00000000006b0e4b in execute_ex (ex=<optimized out>) at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:414 #22 0x00007f382e4ca31f in memtrack_execute_ex (execute_data=0x7f3833c152e0) at /home/tony/php-build/php-src/ext/memtrack/memtrack.c:481 #23 0x00000000006f115d in ZEND_DO_FCALL_SPEC_HANDLER () at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:800 #24 0x00000000006b0e4b in execute_ex (ex=<optimized out>) at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:414 #25 0x00007f382e4ca31f in memtrack_execute_ex (execute_data=0x7f3833c15180) at /home/tony/php-build/php-src/ext/memtrack/memtrack.c:481 #26 0x00000000006f115d in ZEND_DO_FCALL_SPEC_HANDLER () at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:800 #27 0x00000000006b0e4b in execute_ex (ex=<optimized out>) at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:414 #28 0x00007f382e4ca31f in memtrack_execute_ex (execute_data=0x7f3833c14c30) at /home/tony/php-build/php-src/ext/memtrack/memtrack.c:481 (gdb) p *ce Cannot access memory at address 0x7f37a95d414000 (gdb) p ce $1 = (zend_class_entry *) 0x7f37a95d414000 ------------------------------------------------------------------------ [2017-04-10 11:45:23] laruence@php.net Thank you for this bug report. To properly diagnose the problem, we need a backtrace to see what is happening behind the scenes. To find out how to generate a backtrace, please read http://bugs.php.net/bugs-generating-backtrace.php for *NIX and http://bugs.php.net/bugs-generating-backtrace-win32.php for Win32 Once you have generated a backtrace, please submit it to this bug report and change the status back to "Open". Thank you for helping us make PHP better. as we discussed early in mails, I don't think that diretctly relates to that change, anyway, could you please paste the backtrace out? thanks ------------------------------------------------------------------------ [2017-04-10 11:22:45] tony2001@php.net https://bugs.php.net/bug.php?id=73740 - this one looks familiar. ------------------------------------------------------------------------ [2017-04-10 11:17:26] requinix@php.net There have been a few php-fpm crashes reported over the last couple months. Can you tell if any of them are related to this? ------------------------------------------------------------------------ [2017-04-10 11:09:09] tony2001@php.net The following patch has been added/updated: Patch Name: pcre-crashes-fix.diff Revision: 1491822546 URL: https://bugs.php.net/patch-display.php?bug=74403&patch=pcre-crashes-fix.diff&revision=1491822546 ------------------------------------------------------------------------ The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at https://bugs.php.net/bug.php?id=74403 -- Edit this bug report at https://bugs.php.net/bug.php?id=74403&edit=1