Bug #75351 [Opn->Fbk]: Segmentation fault on fatal error during Generator::send

Edit report at https://bugs.php.net/bug.php?id=75351&edit=1 ID: 75351 Updated by: nikic@php.net Reported by: bashofmann at gmail dot com Summary: Segmentation fault on fatal error during Generator::send -Status: Open +Status: Feedback Type: Bug Package: Scripting Engine problem Operating System: Ubuntu 14.04.5 LTS (GNU/Linux 4. PHP Version: 7.1.10 Block user comment: N Private report: N New Comment: I've applied https://github.com/php/php-src/commit/cab0a814bdbfc653754f74b42056c38bdf4fbadb, which fixes some crashes of the type described here I was able to produce under artificial GC configurations. Please check whether current 7.1/7.2/master snapshots resolve the issue for you. I think there is still a lingering problem here, because root and parent references are not necessarily consistent, in that root references might point deeper into the tree than the parent chain under some circumstances. However, I'm not sure if checking for null is the right answer for that (as I think this might miss the generators that are between the root reference and the end of the parent chain) or whether we should be using a reverse loop from root to leaf as we do in most other places. Previous Comments: ------------------------------------------------------------------------ [2018-01-09 14:36:46] daniel dot tschinder at researchgate dot net Did anyone of you find a workaround? We have the same problem, but I cannot really figure out how to workaround and it does not happen every time. I once managed to debug when this happened, but all I saw was that the process died on Generator::send. ------------------------------------------------------------------------ [2017-12-11 23:31:38] tandre@php.net A workaround I'm considering is to call Generator->next(), since my code was already always calling send with NULL. The implementation Generator->send() calls zend_generator_get_current(), but Generator->next() does not. ------------------------------------------------------------------------ [2017-12-11 22:18:03] tandre@php.net I'm encountering similar issues, also in Generator->send(). However, the issues I see are in zend_generator_get_gc instead It seems like garbage collection is getting triggered when Generator->send() is getting called, which may be an edge case that isn't handled properly, and may be related to why it's hard to reproduce. I'm using $retval = yield from, $retval = yield some_helper(), yield other_helper() in the code in question, but the bug happens so infrequently I'm not sure which part it is. I haven't checked how deeply nested yields are. I'm using PHP 7.1.9. The source of zend_generators.c is identical for 7.1.9 and 7.1.12 (latest). Stack trace: ``` ... Lines added by newrelic segfault stack trace dumper omitted /usr/local/php/modules/libphp7.so(+0x431748)[0x7ff795959748] /usr/local/php/modules/libphp7.so(+0x42d4fa)[0x7ff7959554fa] /usr/local/php/modules/libphp7.so(zend_gc_collect_cycles+0x78)[0x7ff795956478] /usr/local/php/modules/libphp7.so(gc_possible_root+0x9e)[0x7ff7959560ee] /usr/local/php/modules/libphp7.so(+0x43220c)[0x7ff79595a20c] /usr/local/php/modules/libphp7.so(zend_objects_store_del+0x271)[0x7ff79596c231] /usr/local/php/modules/libphp7.so(zend_generator_update_current+0x1f0)[0x7ff79595a9b0] /usr/local/php/modules/libphp7.so(zend_generator_resume+0x1dd)[0x7ff79595a4ad] /usr/local/php/modules/libphp7.so(+0x433bdf)[0x7ff79595bbdf] /usr/local/php/modules/libphp7.so(+0x4b70c6)[0x7ff7959df0c6] /usr/local/php/modules/libphp7.so(execute_ex+0x2b)[0x7ff79597738b] /usr/local/php/lib/php/20160303/newrelic.so(+0x231bb)[0x7ff78b0b61bb] /usr/local/php/lib/php/20160303/newrelic.so(+0x23802)[0x7ff78b0b6802] /usr/local/php/modules/libphp7.so(+0x4b6985)[0x7ff7959de985] /usr/local/php/modules/libphp7.so(execute_ex+0x2b)[0x7ff79597738b] /usr/local/php/lib/php/20160303/newrelic.so(+0x231bb)[0x7ff78b0b61bb] /usr/local/php/lib/php/20160303/newrelic.so(+0x23802)[0x7ff78b0b6802] ``` I added comments to parts of the code that showed up in my stack trace, and ``` --- a/Zend/zend_generators.c +++ b/Zend/zend_generators.c @@ -275,7 +275,7 @@ static uint32_t calc_gc_buffer_size(zend_generator *generator) /* {{{ */ /* Yield from root references */ if (generator->node.children == 0) { zend_generator *child = generator, *root = generator->node.ptr.root; - while (root != child) { + while (root != child && child != NULL) { // Not sure if checking for null would solve the underlying issue for the other bug reported, and I'm not familiar with how deep the chain goes. child = child->node.parent; size++; } @@ -340,8 +340,8 @@ static HashTable *zend_generator_get_gc(zval *object, zval **table, int *n) /* { if (generator->node.children == 0) { zend_generator *child = generator, *root = generator->node.ptr.root; - while (root != child) { - child = child->node.parent; + while (root != child && child != NULL) { // Not sure if checking for null would solve the underlying issue + child = child->node.parent; // child->node is somehow equal to null when the segfault occurs ZVAL_OBJ(gc_buffer++, &child->std); } } @@ -608,7 +608,7 @@ ZEND_API zend_generator *zend_generator_update_current(zend_generator *generator } while (!root->execute_data && root != generator) { - OBJ_RELEASE(&old_root->std); + OBJ_RELEASE(&old_root->std); // Not caused by this OBJ_RELEASE. It's caused by the below one. old_root = root; root = zend_generator_get_child(&root->node, leaf); @@ -638,10 +638,11 @@ ZEND_API zend_generator *zend_generator_update_current(zend_generator *generator EG(current_execute_data) = original_execute_data; if (!((old_root ? old_root : generator)->flags & ZEND_GENERATOR_CURRENTLY_RUNNING)) { - leaf->node.ptr.root = root; - root->node.parent = NULL; + leaf->node.ptr.root = root; // The root pointer and root->node pointer were changed. + root->node.parent = NULL; // And a parent gets set to null, which might interfere with zend_generator_get_gc + // What happens if there are nested yield froms, though? Would those still have (something)->node.ptr.root point to old_root? if (old_root) { - OBJ_RELEASE(&old_root->std); + OBJ_RELEASE(&old_root->std); // Disassembly shows that this triggers a garbage collection } zend_generator_resume(leaf); return leaf->node.ptr.root; /* this may be updated during zend_generator_resume! */ ``` ------------------------------------------------------------------------ [2017-10-10 15:15:27] bashofmann at gmail dot com Description: ------------ We are experiencing segmentation faults related to probably fatal errors when using Generator::send. Unfortunately I was not able to create a simple script to reproduce this issue. gdb back trace: (gdb) bt #0 calc_gc_buffer_size (generator=0x7f64d68f4a80) at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_generators.c:279 #1 zend_generator_get_gc (object=0x7ffe4c963eb0, table=0x7ffe4c963ea8, n=0x7ffe4c963ea4) at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_generators.c:305 #2 0x00005576280aa23a in gc_mark_grey (ref=0x7f64d68f4a80) at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_gc.c:499 #3 0x00005576280aa2b6 in gc_mark_grey (ref=<optimized out>) at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_gc.c:511 #4 0x00005576280aa2b6 in gc_mark_grey (ref=<optimized out>) at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_gc.c:511 #5 0x00005576280aa346 in gc_mark_grey (ref=<optimized out>) at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_gc.c:572 #6 0x00005576280aa346 in gc_mark_grey (ref=<optimized out>) at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_gc.c:572 #7 0x00005576280ab1e8 in gc_mark_roots () at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_gc.c:598 #8 zend_gc_collect_cycles () at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_gc.c:1072 #9 0x00005576280aae3e in gc_possible_root (ref=0x7ffe4c963eb0, ref@entry=0x7f64d68f5f80) at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_gc.c:286 #10 0x00005576280af551 in zend_object_release (obj=0x7f64d68f5f80) at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_objects_API.h:80 #11 zend_generator_update_current (generator=generator@entry=0x7f64d68f4a80, leaf=0x7f64d68f4a80) at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_generators.c:667 #12 0x00005576280aefdd in zend_generator_get_current (generator=0x7f64d68f4a80) at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_generators.h:133 #13 zend_generator_resume (orig_generator=orig_generator@entry=0x7f64d68f4a80) at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_generators.c:845 #14 0x00005576280afe7f in zim_Generator_send (execute_data=0x7f6555a14070, return_value=0x7ffe4c964210) at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_generators.c:1001 #15 0x000055762811a0cc in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:972 #16 0x00005576280c8edb in execute_ex (ex=<optimized out>) at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:429 #17 0x0000557628119f65 in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:949 #18 0x00005576280c8edb in execute_ex (ex=<optimized out>) at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:429 #19 0x0000557628119f65 in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:949 #20 0x00005576280c8edb in execute_ex (ex=<optimized out>) at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:429 #21 0x0000557628119f65 in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:949 #22 0x00005576280c8edb in execute_ex (ex=<optimized out>) at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:429 #23 0x0000557628119f65 in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:949 #24 0x00005576280c8edb in execute_ex (ex=<optimized out>) at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:429 #25 0x0000557628119ae5 in ZEND_DO_FCALL_SPEC_RETVAL_USED_HANDLER () at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:1076 #26 0x00005576280c8edb in execute_ex (ex=<optimized out>) at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:429 #27 0x0000557628119ae5 in ZEND_DO_FCALL_SPEC_RETVAL_USED_HANDLER () at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:1076 #28 0x00005576280c8edb in execute_ex (ex=<optimized out>) at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:429 #29 0x0000557628119f65 in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:949 #30 0x00005576280c8edb in execute_ex (ex=<optimized out>) at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:429 #31 0x00005576280aeede in zend_generator_resume (orig_generator=0x7f64de060780) at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_generators.c:815 #32 0x000055762811a0cc in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:972 #33 0x00005576280c8edb in execute_ex (ex=<optimized out>) at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:429 #34 0x0000557628119f65 in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:949 #35 0x00005576280c8edb in execute_ex (ex=<optimized out>) at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:429 #36 0x0000557628119ae5 in ZEND_DO_FCALL_SPEC_RETVAL_USED_HANDLER () at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:1076 #37 0x00005576280c8edb in execute_ex (ex=<optimized out>) at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:429 #38 0x00005576280747e1 in zend_call_function (fci=fci@entry=0x7ffe4c964810, fci_cache=fci_cache@entry=0x7ffe4c9647e0) at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_execute_API.c:855 #39 0x0000557627f762ad in reflection_method_invoke (execute_data=<optimized out>, return_value=0x7f6555a13610, variadic=0) at /build/php7.1-zUmk8m/php7.1-7.1.9/ext/reflection/php_reflection.c:3342 #40 0x0000557628119c50 in ZEND_DO_FCALL_SPEC_RETVAL_USED_HANDLER () at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:1099 #41 0x00005576280c8edb in execute_ex (ex=<optimized out>) at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:429 #42 0x0000557628119ae5 in ZEND_DO_FCALL_SPEC_RETVAL_USED_HANDLER () at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:1076 #43 0x00005576280c8edb in execute_ex (ex=<optimized out>) at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:429 #44 0x0000557628119f65 in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:949 #45 0x00005576280c8edb in execute_ex (ex=<optimized out>) at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:429 #46 0x0000557628119f65 in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:949 #47 0x00005576280c8edb in execute_ex (ex=<optimized out>) at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:429 #48 0x0000557628119f65 in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:949 #49 0x00005576280c8edb in execute_ex (ex=<optimized out>) at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:429 #50 0x0000557628119f65 in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:949 #51 0x00005576280c8edb in execute_ex (ex=<optimized out>) at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:429 #52 0x0000557628119f65 in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:949 #53 0x00005576280c8edb in execute_ex (ex=<optimized out>) at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:429 #54 0x000055762811bf74 in zend_execute (op_array=0x7f6555a67000, op_array@entry=0x7f64f0edbe88, return_value=return_value@entry=0x7f6555a13550) at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend_vm_execute.h:474 #55 0x0000557628083e44 in zend_execute_scripts (type=type@entry=8, retval=0x7f6555a13550, retval@entry=0x0, file_count=file_count@entry=3) at /build/php7.1-zUmk8m/php7.1-7.1.9/Zend/zend.c:1480 #56 0x00005576280234a8 in php_execute_script (primary_file=0x7ffe4c966f90) at /build/php7.1-zUmk8m/php7.1-7.1.9/main/main.c:2552 #57 0x0000557627f0c4ac in main (argc=<optimized out>, argv=<optimized out>) at /build/php7.1-zUmk8m/php7.1-7.1.9/sapi/fpm/fpm/fpm_main.c:1966 Excerpt of zbacktrace (gdb) zbacktrace [0x7f6555a14070] Generator->send(object[0x7f6555a140c0]) [internal function] [0x7f6555a13fd0] rg\core\pow\preparer\handler\GeneratorHandler->returnResponse(object[0x7f6555a14020], object[0x7f6555a14030], reference) /home/researchgate/community/tor_prd/blue/src/core/pow/preparer/handler/GeneratorHandler.php:60 [0x7f6555a13f40] rg\core\pow\preparer\handler\PreparableHandler->returnResponse(object[0x7f6555a13f90], object[0x7f6555a13fa0], array(1)[0x7f6555a13fb0]) /home/researchgate/community/tor_prd/blue/src/core/pow/preparer/handler/PreparableHandler.php:82 ... the code of returnResponse in GeneratorHandler looks rather innocent: public function returnResponse($preparable, \Traversable $iterator, array $data) { if (!is_array($iterator->rawCurrent()) && count($data) === 1) { $data = reset($data); } $preparable->send($data); } The reason why I'm suspecting it has something to do with a fatal error is that it seems to go through this line https://github.com/php/php-src/blob/PHP-7.1.9/Zend/zend_generators.c#L133 in the bt ------------------------------------------------------------------------ -- Edit this bug report at https://bugs.php.net/bug.php?id=75351&edit=1