Bug #65200 [Com]: Seg faults in php_free_pcre_cache on child exit

Edit report at https://bugs.php.net/bug.php?id=65200&edit=1 ID: 65200 Comment by: mmucklo at corp dot oodle dot com Reported by: mmucklo at corp dot oodle dot com Summary: Seg faults in php_free_pcre_cache on child exit Status: Open Type: Bug Package: *General Issues Operating System: RHEL 6.4 PHP Version: 5.4.16 Block user comment: N Private report: N New Comment: Okay, I recompiled without --enable-debug, and pounded the server and got two core files that are exactly the same backtrace-wise... Program terminated with signal 11, Segmentation fault. #0 _zend_mm_free_int (heap=0x1bea970, p=0x7f1393ded878) at /workspace/source/external/build/php-5.4.17-apache/Zend/zend_alloc.c:2100 2100 /workspace/source/external/build/php-5.4.17-apache/Zend/zend_alloc.c: No such file or directory. in /workspace/source/external/build/php-5.4.17-apache/Zend/zend_alloc.c (gdb) bt #0 _zend_mm_free_int (heap=0x1bea970, p=0x7f1393ded878) at /workspace/source/external/build/php-5.4.17-apache/Zend/zend_alloc.c:2100 #1 0x00007f13b8619e65 in destroy_op_array (op_array=0x28ba910) at /workspace/source/external/build/php-5.4.17- apache/Zend/zend_opcode.c:364 #2 0x00007f13b863062b in zend_hash_destroy (ht=0x28b9748) at /workspace/source/external/build/php-5.4.17-apache/Zend/zend_hash.c:560 #3 0x00007f13b861a22e in destroy_zend_class (pce=<value optimized out>) at /workspace/source/external/build/php-5.4.17- apache/Zend/zend_opcode.c:296 #4 0x00007f13b86302b5 in zend_hash_apply_deleter (ht=0x1beb2d0, p=0x27a7a10) at /workspace/source/external/build/php-5.4.17- apache/Zend/zend_hash.c:650 #5 0x00007f13b86303c9 in zend_hash_reverse_apply (ht=0x1beb2d0, apply_func=0x7f13b8614c30 <clean_non_persistent_class>) at /workspace/source/external/build/php-5.4.17-apache/Zend/zend_hash.c:804 #6 0x00007f13b8618486 in shutdown_executor () at /workspace/source/external/build/php-5.4.17-apache/Zend/zend_execute_API.c:303 #7 0x00007f13b86239e2 in zend_deactivate () at /workspace/source/external/build/php-5.4.17-apache/Zend/zend.c:938 #8 0x00007f13b85bff3b in php_request_shutdown (dummy=<value optimized out>) at /workspace/source/external/build/php-5.4.17- apache/main/main.c:1800 #9 0x00007f13b86d08e7 in php_apache_request_dtor (r=0x30de620) at /workspace/source/external/build/php-5.4.17- apache/sapi/apache2handler/sapi_apache2.c:507 #10 php_handler (r=0x30de620) at /workspace/source/external/build/php-5.4.17-apache/sapi/apache2handler/sapi_apache2.c:679 #11 0x0000000000441d20 in ap_run_handler (r=0x30de620) at config.c:158 #12 0x000000000044534e in ap_invoke_handler (r=0x30de620) at config.c:376 #13 0x000000000048c180 in ap_process_request (r=0x30de620) at http_request.c:282 #14 0x0000000000489140 in ap_process_http_connection (c=0x219b640) at http_core.c:190 #15 0x00000000004492b0 in ap_run_process_connection (c=0x219b640) at connection.c:43 #16 0x00000000004b9078 in child_main (child_num_arg=<value optimized out>) at prefork.c:667 #17 0x00000000004b9374 in make_child (s=0x1a78c80, slot=14) at prefork.c:768 #18 0x00000000004b9fc7 in perform_idle_server_maintenance (_pconf=<value optimized out>, plog=<value optimized out>, s=<value optimized out>) at prefork.c:903 #19 ap_mpm_run (_pconf=<value optimized out>, plog=<value optimized out>, s=<value optimized out>) at prefork.c:1107 #20 0x000000000042e564 in main (argc=3, argv=0x7fff7846b3c8) at main.c:753 Core was generated by `/service/local/apache/bin/httpd -f /service/conf/httpd.qvc.conf'. Program terminated with signal 11, Segmentation fault. #0 _zend_mm_free_int (heap=0x1bea970, p=0x7f1393ded878) at /workspace/source/external/build/php-5.4.17-apache/Zend/zend_alloc.c:2100 2100 /workspace/source/external/build/php-5.4.17-apache/Zend/zend_alloc.c: No such file or directory. in /workspace/source/external/build/php-5.4.17-apache/Zend/zend_alloc.c (gdb) bt #0 _zend_mm_free_int (heap=0x1bea970, p=0x7f1393ded878) at /workspace/source/external/build/php-5.4.17-apache/Zend/zend_alloc.c:2100 #1 0x00007f13b8619e65 in destroy_op_array (op_array=0x5953800) at /workspace/source/external/build/php-5.4.17- apache/Zend/zend_opcode.c:364 #2 0x00007f13b863062b in zend_hash_destroy (ht=0x5a51cc8) at /workspace/source/external/build/php-5.4.17-apache/Zend/zend_hash.c:560 #3 0x00007f13b861a22e in destroy_zend_class (pce=<value optimized out>) at /workspace/source/external/build/php-5.4.17- apache/Zend/zend_opcode.c:296 #4 0x00007f13b86302b5 in zend_hash_apply_deleter (ht=0x1beb2d0, p=0x4e33d70) at /workspace/source/external/build/php-5.4.17- apache/Zend/zend_hash.c:650 #5 0x00007f13b86303c9 in zend_hash_reverse_apply (ht=0x1beb2d0, apply_func=0x7f13b8614c30 <clean_non_persistent_class>) at /workspace/source/external/build/php-5.4.17-apache/Zend/zend_hash.c:804 #6 0x00007f13b8618486 in shutdown_executor () at /workspace/source/external/build/php-5.4.17-apache/Zend/zend_execute_API.c:303 #7 0x00007f13b86239e2 in zend_deactivate () at /workspace/source/external/build/php-5.4.17-apache/Zend/zend.c:938 #8 0x00007f13b85bff3b in php_request_shutdown (dummy=<value optimized out>) at /workspace/source/external/build/php-5.4.17- apache/main/main.c:1800 #9 0x00007f13b86d08e7 in php_apache_request_dtor (r=0x21a94c0) at /workspace/source/external/build/php-5.4.17- apache/sapi/apache2handler/sapi_apache2.c:507 #10 php_handler (r=0x21a94c0) at /workspace/source/external/build/php-5.4.17-apache/sapi/apache2handler/sapi_apache2.c:679 #11 0x0000000000441d20 in ap_run_handler (r=0x21a94c0) at config.c:158 #12 0x000000000044534e in ap_invoke_handler (r=0x21a94c0) at config.c:376 #13 0x000000000048c180 in ap_process_request (r=0x21a94c0) at http_request.c:282 #14 0x0000000000489140 in ap_process_http_connection (c=0x219b640) at http_core.c:190 #15 0x00000000004492b0 in ap_run_process_connection (c=0x219b640) at connection.c:43 #16 0x00000000004b9078 in child_main (child_num_arg=<value optimized out>) at prefork.c:667 #17 0x00000000004b9374 in make_child (s=0x1a78c80, slot=4) at prefork.c:768 #18 0x00000000004b967e in startup_children (_pconf=<value optimized out>, plog=<value optimized out>, s=<value optimized out>) at prefork.c:786 #19 ap_mpm_run (_pconf=<value optimized out>, plog=<value optimized out>, s=<value optimized out>) at prefork.c:1007 #20 0x000000000042e564 in main (argc=3, argv=0x7fff7846b3c8) at main.c:753 (gdb) quit Previous Comments: ------------------------------------------------------------------------ [2013-07-17 06:48:16] ab@php.net Yes, i meant PHP --enable-debug. That's a bit tricky, with gcc using --enable- debug directly will define some macros which switch the code parts. Using no -- enable-debug one still can enforce debug symbols using sometihng like CFLAGS="-ggdb -O3" CXXFLAGS="$CFLAGS" ./configure That's why i meant it might be not reproduceable without --enable-debug as the code corresponding to the last BT is different then. Despite all that, i think the first PCRE backtrace is a real bug. ------------------------------------------------------------------------ [2013-07-16 17:47:37] mmucklo at corp dot oodle dot com I'm not understanding you about the "debug" build part? I don't think it gives a backtrace without --enable-debug, unless you are referring to apache being compiled in Debug mode as well... ------------------------------------------------------------------------ [2013-07-16 11:52:43] ab@php.net Yep, the last BT is very far from the first one. It even doesn't mention PCRE at all. But nevertheless, seems the last BT is done with a debug build. Was there something interesting on stderr? Also the last BT might be not reproducable with a release build. I'm stuck reproducing your first BT, sadly. ------------------------------------------------------------------------ [2013-07-15 22:46:00] mmucklo at corp dot oodle dot com Upgraded to PHP 5.4.17, fewer cores, it seems, but still seeing a couple so far... The backtrace has changed, though: ---------------------------------- Core was generated by `/service/local/apache/bin/httpd -f /service/conf/httpd.qvc.conf'. Program terminated with signal 11, Segmentation fault. #0 __memcmp_sse2 () at ../sysdeps/x86_64/memcmp.S:57 57 movl (%rdi), %eax (gdb) bt #0 __memcmp_sse2 () at ../sysdeps/x86_64/memcmp.S:57 #1 0x00007f15016adb0a in zend_mm_check_ptr (heap=0xdc3a10, ptr=0x7f14dc01e2a8, silent=0, __zend_filename=0x7f1501c250a8 "/workspace/source/external/build/php- 5.4.17-apache/Zend/zend_opcode.c", __zend_lineno=364, __zend_orig_filename=0x0, __zend_orig_lineno=0) at /workspace/source/external/build/php-5.4.17-apache/Zend/zend_alloc.c:1515 #2 0x00007f15016ad65a in zend_mm_check_ptr (heap=0xdc3a10, ptr=0x7f14dc01e2a8, silent=1, __zend_filename=0x7f1501c250a8 "/workspace/source/external/build/php- 5.4.17-apache/Zend/zend_opcode.c", __zend_lineno=364, __zend_orig_filename=0x0, __zend_orig_lineno=0) at /workspace/source/external/build/php-5.4.17-apache/Zend/zend_alloc.c:1416 #3 0x00007f15016af182 in _zend_mm_free_int (heap=0xdc3a10, p=0x7f14dc01e2a8, __zend_filename=0x7f1501c250a8 "/workspace/source/external/build/php-5.4.17- apache/Zend/zend_opcode.c", __zend_lineno=364, __zend_orig_filename=0x0, __zend_orig_lineno=0) at /workspace/source/external/build/php-5.4.17-apache/Zend/zend_alloc.c:2064 #4 0x00007f15016b080d in _efree (ptr=0x7f14dc01e2a8, __zend_filename=0x7f1501c250a8 "/workspace/source/external/build/php-5.4.17- apache/Zend/zend_opcode.c", __zend_lineno=364, __zend_orig_filename=0x0, __zend_orig_lineno=0) at /workspace/source/external/build/php-5.4.17- apache/Zend/zend_alloc.c:2436 #5 0x00007f15016d95a1 in destroy_op_array (op_array=0x1470ab8) at /workspace/source/external/build/php-5.4.17-apache/Zend/zend_opcode.c:364 #6 0x00007f15016d890f in destroy_zend_function (function=0x1470ab8) at /workspace/source/external/build/php-5.4.17-apache/Zend/zend_opcode.c:112 #7 0x00007f15016d8929 in zend_function_dtor (function=0x1470ab8) at /workspace/source/external/build/php-5.4.17-apache/Zend/zend_opcode.c:124 #8 0x00007f15016f9ecf in zend_hash_destroy (ht=0x1470698) at /workspace/source/external/build/php-5.4.17-apache/Zend/zend_hash.c:560 #9 0x00007f15016d923d in destroy_zend_class (pce=0x3586b48) at /workspace/source/external/build/php-5.4.17-apache/Zend/zend_opcode.c:296 #10 0x00007f15016fa2bc in zend_hash_apply_deleter (ht=0xdc4370, p=0x3586b30) at /workspace/source/external/build/php-5.4.17-apache/Zend/zend_hash.c:650 #11 0x00007f15016fa979 in zend_hash_reverse_apply (ht=0xdc4370, apply_func=0x7f15016d200e <clean_non_persistent_class>) at /workspace/source/external/build/php-5.4.17-apache/Zend/zend_hash.c:804 #12 0x00007f15016d287e in shutdown_executor () at /workspace/source/external/build/php-5.4.17-apache/Zend/zend_execute_API.c:303 #13 0x00007f15016e6e56 in zend_deactivate () at /workspace/source/external/build/php-5.4.17-apache/Zend/zend.c:938 #14 0x00007f1501652b03 in php_request_shutdown (dummy=0x0) at /workspace/source/external/build/php-5.4.17-apache/main/main.c:1800 #15 0x00007f1501792808 in php_apache_request_dtor (r=0x1383150) at /workspace/source/external/build/php-5.4.17- apache/sapi/apache2handler/sapi_apache2.c:507 #16 0x00007f1501793079 in php_handler (r=0x1383150) at /workspace/source/external/build/php-5.4.17- apache/sapi/apache2handler/sapi_apache2.c:679 #17 0x0000000000441d90 in ap_run_handler (r=0x1383150) at config.c:158 #18 0x00000000004453ee in ap_invoke_handler (r=0x1383150) at config.c:376 #19 0x000000000048ca60 in ap_process_request (r=0x1383150) at http_request.c:282 #20 0x0000000000489a08 in ap_process_http_connection (c=0x13752d0) at http_core.c:190 #21 0x0000000000449330 in ap_run_process_connection (c=0x13752d0) at connection.c:43 #22 0x00000000004b9bc8 in child_main (child_num_arg=<value optimized out>) at prefork.c:667 #23 0x00000000004b9ec4 in make_child (s=0xc51c80, slot=16) at prefork.c:768 #24 0x00000000004bab17 in perform_idle_server_maintenance (_pconf=<value optimized out>, plog=<value optimized out>, s=<value optimized out>) at prefork.c:903 #25 ap_mpm_run (_pconf=<value optimized out>, plog=<value optimized out>, s= <value optimized out>) at prefork.c:1107 #26 0x000000000042e524 in main (argc=3, argv=0x7fffd6d0d018) at main.c:753 ------------------------------------------------------------------------ [2013-07-08 07:16:05] ab@php.net From what i could analyse yet, this issue locale related. It happens on prefork child shutdown, so also PHP module shutdown. From this point not very bad. The PCRE patterns are cached by locale, so it might be reproduceable with a scenario like this - set locale - do some pcre stuff - change locale - do some pcre stuff ....... This is most likely a race condition in MSHUTDOWN while freeing PCRE cache under Apache prefork. You could try a simple script with this scenario. I'll be doing the same in the meantime. Or maybe you recognize this pattern in your app? Unfortunately that's all I could read from the BT so far. Thanks ------------------------------------------------------------------------ The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at https://bugs.php.net/bug.php?id=65200 -- Edit this bug report at https://bugs.php.net/bug.php?id=65200&edit=1