Bug #74403 [Asn->Fbk]: Sporadic FPM segfaults

Edit report at https://bugs.php.net/bug.php?id=74403&edit=1 ID: 74403 Updated by: cmb@php.net Reported by: tony2001@php.net Summary: Sporadic FPM segfaults -Status: Assigned +Status: Feedback Type: Bug Package: PCRE related Operating System: Linux PHP Version: 7.1.4RC1 -Assigned To: laruence +Assigned To: cmb Block user comment: N Private report: N New Comment: Is this still an issue with any of the actively supported PHP versions[1]? [1] <https://www.php.net/supported-versions.php> Previous Comments: ------------------------------------------------------------------------ [2017-04-14 02:52:14] laruence@php.net could you try to understand what the different did your patch makes? I will try to explain it again: you patch change : if (!ZSTR_IS_INTERNED(key) || !(GC_FLAGS(key) & IS_STR_PERMANENT)) { pce = zend_hash_str_update_mem(&PCRE_G(pcre_cache), ZSTR_VAL(key), ZSTR_LEN(key), &new_entry, sizeof(pcre_cache_entry)); #if HAVE_SETLOCALE if (key != regex) { zend_string_release(key); } #endif } else { pce = zend_hash_update_mem(&PCRE_G(pcre_cache), key, &new_entry, sizeof(pcre_cache_entry)); } -------------------------------------------------------------------------------------------------- to: if (!ZSTR_IS_INTERNED(regex) || !(GC_FLAGS(regex) & IS_STR_PERMANENT)) { zend_string *str = zend_string_init(ZSTR_VAL(regex), ZSTR_LEN(regex), 1); GC_REFCOUNT(str) = 0; /* will be incremented by zend_hash_update_mem() */ ZSTR_H(str) = ZSTR_H(regex); regex = str; } pce = zend_hash_update_mem(&PCRE_G(pcre_cache), regex, &new_entry, sizeof(pcre_cache_entry)); ---------------------------------- but to me, they are actually doing the exactlly same thing pce = zend_hash_str_update_mem(&PCRE_G(pcre_cache), ZSTR_VAL(key), ZSTR_LEN(key), &new_entry, sizeof(pcre_cache_entry)); will allocated a persistent key for str_key because &PCRG_G(pcre_cache) is a persistent hashtable so, as I said, your patch may cover some real problems here. please try to disable memtrack ------------------------------------------------------------------------ [2017-04-13 11:55:44] tony2001@php.net > from the backtrace, it's really not like a problem that is the "key change" > caused That's why I called the segfaults "sporadic". They happen in random places, apparently because the interned string used for the hash gets broken in the process. ------------------------------------------------------------------------ [2017-04-13 02:57:40] laruence@php.net from the backtrace, it's really not like a problem that is the "key change" caused could you try without memtrack ? anyway, @tony2001, I understand your concern, but we should try to find the root cause, you patch may wroks, but it also may cover some real problems. ------------------------------------------------------------------------ [2017-04-11 09:51:42] tony2001@php.net Reverted the attached patch in my private branch and deployed FPM on one production server. After about 6 hours I've got several cores with the backtraces below. Number of segfaults on several hundred servers since March 22: 0. backtrace #1: #0 zend_hash_find_bucket (ht=ht@entry=0xdd28e1cadaf9342c, ht=ht@entry=0xdd28e1cadaf9342c, key=0x7f37b66bdf00, key@entry=0x7f57b453b718) at /home/tony/php-build/php-src/Zend/zend_hash.c:496 #1 zend_hash_find (ht=ht@entry=0x7f37b4539720, key=key@entry=0x7f37b66bdf00) at /home/tony/php-build/php-src/Zend/zend_hash.c:1988 #2 0x00000000006da2e6 in zend_hash_find_ind (key=0x7f37b66bdf00, ht=0x7f37b4539720) at /home/tony/php-build/php-src/Zend/zend_hash.h:278 #3 ZEND_ISSET_ISEMPTY_DIM_OBJ_SPEC_CV_CV_HANDLER () at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:37856 #4 0x00000000006b0e4b in execute_ex (ex=<optimized out>) at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:414 #5 0x00007f382e4ca31f in memtrack_execute_ex (execute_data=0x7f3833c18720) at /home/tony/php-build/php-src/ext/memtrack/memtrack.c:481 #6 0x00000000006f115d in ZEND_DO_FCALL_SPEC_HANDLER () at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:800 #7 0x00000000006b0e4b in execute_ex (ex=<optimized out>) at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:414 #8 0x00007f382e4ca31f in memtrack_execute_ex (execute_data=0x7f3833c18660) at /home/tony/php-build/php-src/ext/memtrack/memtrack.c:481 #9 0x00000000006f115d in ZEND_DO_FCALL_SPEC_HANDLER () at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:800 #10 0x00000000006b0e4b in execute_ex (ex=<optimized out>) at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:414 #11 0x00007f382e4ca31f in memtrack_execute_ex (execute_data=0x7f3833c18480) at /home/tony/php-build/php-src/ext/memtrack/memtrack.c:481 #12 0x000000000065b919 in zend_call_function (fci=fci@entry=0x7ffcffa5d8d0, fci_cache=fci_cache@entry=0x7ffcffa5d8a0) at /home/tony/php-build/php-src/Zend/zend_execute_API.c:858 #13 0x0000000000689db3 in zend_call_method (object=0x0, obj_ce=<optimized out>, fn_proxy=0x7f3833c55870, function_name=0x7f3833c74248 "main_autoload", function_name_len=<optimized out>, retval_ptr=retval_ptr@entry=0x0, param_count=1, arg1=0x7f3833c18470, arg2=0x0) at /home/tony/php-build/php-src/Zend/zend_interfaces.c:104 #14 0x0000000000525c4e in zif_spl_autoload_call (execute_data=<optimized out>, return_value=<optimized out>) at /home/tony/php-build/php-src/ext/spl/php_spl.c:421 #15 0x00007f382e4ca680 in memtrack_execute_internal (current_execute_data=0x7f3833c18410, return_value=0x7ffcffa5dbc0) at /home/tony/php-build/php-src/ext/memtrack/memtrack.c:563 #16 0x000000000065bb57 in zend_call_function (fci=fci@entry=0x7ffcffa5dc00, fci_cache=fci_cache@entry=0x7ffcffa5dbd0) at /home/tony/php-build/php-src/Zend/zend_execute_API.c:880 #17 0x000000000065bee2 in zend_lookup_class_ex (name=name@entry=0x7f37b66bdf00, key=key@entry=0x0, use_autoload=use_autoload@entry=1) at /home/tony/php-build/php-src/Zend/zend_execute_API.c:1040 #18 0x000000000065c635 in zend_fetch_class (class_name=0x7f37b66bdf00, fetch_type=512) at /home/tony/php-build/php-src/Zend/zend_execute_API.c:1364 #19 0x00000000006e3d2a in ZEND_FETCH_CLASS_SPEC_CV_HANDLER () at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:2330 #20 0x00000000006b0e4b in execute_ex (ex=<optimized out>) at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:414 #21 0x00007f382e4ca31f in memtrack_execute_ex (execute_data=0x7f3833c18070) at /home/tony/php-build/php-src/ext/memtrack/memtrack.c:481 #22 0x00000000006f115d in ZEND_DO_FCALL_SPEC_HANDLER () at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:800 #23 0x00000000006b0e4b in execute_ex (ex=<optimized out>) at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:414 #24 0x00007f382e4ca31f in memtrack_execute_ex (execute_data=0x7f3833c14e60) at /home/tony/php-build/php-src/ext/memtrack/memtrack.c:481 #25 0x00000000006f115d in ZEND_DO_FCALL_SPEC_HANDLER () at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:800 #26 0x00000000006b0e4b in execute_ex (ex=<optimized out>) at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:414 #27 0x00007f382e4ca31f in memtrack_execute_ex (execute_data=0x7f3833c134c0) at /home/tony/php-build/php-src/ext/memtrack/memtrack.c:481 #28 0x00000000006f115d in ZEND_DO_FCALL_SPEC_HANDLER () at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:800 (gdb) p p $1 = (Bucket *) 0x7f57b453b718 (gdb) p *p Cannot access memory at address 0x7f57b453b718 backtrace #2: #0 zend_verify_abstract_class (ce=0x7f37a95d414000) at /home/tony/php-build/php-src/Zend/zend_execute_API.c:1441 #1 0x00000000006b6f24 in ZEND_VERIFY_ABSTRACT_CLASS_SPEC_HANDLER () at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:1539 #2 0x00000000006b0e4b in execute_ex (ex=<optimized out>) at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:414 #3 0x00007f382e4ca31f in memtrack_execute_ex (execute_data=0x7f3833c15af0) at /home/tony/php-build/php-src/ext/memtrack/memtrack.c:481 #4 0x000000000071673c in ZEND_INCLUDE_OR_EVAL_SPEC_CV_HANDLER () at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:29532 #5 0x00000000006b0e4b in execute_ex (ex=<optimized out>) at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:414 #6 0x00007f382e4ca31f in memtrack_execute_ex (execute_data=0x7f3833c15720) at /home/tony/php-build/php-src/ext/memtrack/memtrack.c:481 #7 0x00000000006f115d in ZEND_DO_FCALL_SPEC_HANDLER () at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:800 #8 0x00000000006b0e4b in execute_ex (ex=<optimized out>) at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:414 #9 0x00007f382e4ca31f in memtrack_execute_ex (execute_data=0x7f3833c156a0) at /home/tony/php-build/php-src/ext/memtrack/memtrack.c:481 #10 0x00000000006f115d in ZEND_DO_FCALL_SPEC_HANDLER () at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:800 #11 0x00000000006b0e4b in execute_ex (ex=<optimized out>) at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:414 #12 0x00007f382e4ca31f in memtrack_execute_ex (execute_data=0x7f3833c154c0) at /home/tony/php-build/php-src/ext/memtrack/memtrack.c:481 #13 0x000000000065b919 in zend_call_function (fci=fci@entry=0x7ffcffa5d0c0, fci_cache=fci_cache@entry=0x7ffcffa5d090) at /home/tony/php-build/php-src/Zend/zend_execute_API.c:858 #14 0x0000000000689db3 in zend_call_method (object=0x0, obj_ce=<optimized out>, fn_proxy=0x7f3833c55810, function_name=0x7f3833c762e8 "main_autoload", function_name_len=<optimized out>, retval_ptr=retval_ptr@entry=0x0, param_count=1, arg1=0x7f3833c154b0, arg2=0x0) at /home/tony/php-build/php-src/Zend/zend_interfaces.c:104 #15 0x0000000000525c4e in zif_spl_autoload_call (execute_data=<optimized out>, return_value=<optimized out>) at /home/tony/php-build/php-src/ext/spl/php_spl.c:421 #16 0x00007f382e4ca680 in memtrack_execute_internal (current_execute_data=0x7f3833c15450, return_value=0x7ffcffa5d3b0) at /home/tony/php-build/php-src/ext/memtrack/memtrack.c:563 #17 0x000000000065bb57 in zend_call_function (fci=fci@entry=0x7ffcffa5d3f0, fci_cache=fci_cache@entry=0x7ffcffa5d3c0) at /home/tony/php-build/php-src/Zend/zend_execute_API.c:880 #18 0x000000000065bee2 in zend_lookup_class_ex (name=name@entry=0x7f37b4487cd0, key=0x7f37b4487b98, use_autoload=use_autoload@entry=1) at /home/tony/php-build/php-src/Zend/zend_execute_API.c:1040 #19 0x000000000065c7d8 in zend_fetch_class_by_name (class_name=0x7f37b4487cd0, key=<optimized out>, fetch_type=fetch_type@entry=512) at /home/tony/php-build/php-src/Zend/zend_execute_API.c:1386 #20 0x00000000007148d7 in ZEND_INIT_STATIC_METHOD_CALL_SPEC_CONST_CONST_HANDLER () at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:5667 #21 0x00000000006b0e4b in execute_ex (ex=<optimized out>) at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:414 #22 0x00007f382e4ca31f in memtrack_execute_ex (execute_data=0x7f3833c152e0) at /home/tony/php-build/php-src/ext/memtrack/memtrack.c:481 #23 0x00000000006f115d in ZEND_DO_FCALL_SPEC_HANDLER () at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:800 #24 0x00000000006b0e4b in execute_ex (ex=<optimized out>) at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:414 #25 0x00007f382e4ca31f in memtrack_execute_ex (execute_data=0x7f3833c15180) at /home/tony/php-build/php-src/ext/memtrack/memtrack.c:481 #26 0x00000000006f115d in ZEND_DO_FCALL_SPEC_HANDLER () at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:800 #27 0x00000000006b0e4b in execute_ex (ex=<optimized out>) at /home/tony/php-build/php-src/Zend/zend_vm_execute.h:414 #28 0x00007f382e4ca31f in memtrack_execute_ex (execute_data=0x7f3833c14c30) at /home/tony/php-build/php-src/ext/memtrack/memtrack.c:481 (gdb) p *ce Cannot access memory at address 0x7f37a95d414000 (gdb) p ce $1 = (zend_class_entry *) 0x7f37a95d414000 ------------------------------------------------------------------------ The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at https://bugs.php.net/bug.php?id=74403 -- Edit this bug report at https://bugs.php.net/bug.php?id=74403&edit=1