Bug #77955 [Com]: Random segmentation fault in mysqlnd from php-fpm

Edit report at https://bugs.php.net/bug.php?id=77955&edit=1 ID: 77955 Comment by: victoria at sharksmedia dot dk Reported by: victoria at sharksmedia dot dk Summary: Random segmentation fault in mysqlnd from php-fpm Status: Assigned Type: Bug Package: PDO MySQL Operating System: CentOS 7.6.1810 PHP Version: 7.3.4 Assigned To: dmitry Block user comment: N Private report: N New Comment: (gdb) up #1 mysqlnd_mysqlnd_res_meta_free_pub (meta=0x7f492977a718) at /usr/src/debug/php-7.3.4/ext/mysqlnd/mysqlnd_result_meta.c:106 106 php_mysqlnd_free_field_metadata(fields++); (gdb) p *meta $1 = {fields = 0x7f4929401018, m = 0x7f49394e1080 <mysqlnd_mysqlnd_res_meta_methods>, current_field = 0, field_count = 159} (gdb) p i $2 = <optimized out> (gdb) p meta.fields[0] Cannot access memory at address 0x7f4929401018 (gdb) p meta.fields[i-1] value has been optimized out (gdb) Previous Comments: ------------------------------------------------------------------------ [2019-05-07 13:14:31] dmitry@php.net Few more commands. (gdb) up (gdb) p *meta (gdb) p i (gdb) p meta.fields[0] (gdb) p meta.fields[i-1] Probably, "meta.fields" was somehow corrupted or used after free, but I can't guess how this might happen just analysing sources. I need a way to reproduce this, to catch the problem. ------------------------------------------------------------------------ [2019-05-07 13:05:14] nikic@php.net My theory here would be that we need to swap the free_result_buffer() and free_metadata() calls inside https://github.com/php/php-src/blob/master/ext/mysqlnd/mysqlnd_result.c#L293, because free_result_internal() will destroy the mempool, which is shared with the metadata. Does that sound plausible Dmitry? ------------------------------------------------------------------------ [2019-05-07 12:52:02] Scott at exussum dot co dot uk Not sure if it helps but we have a long running job which gets this same issue. It happens at a different point each time. My guess was it was something to do with gc but not proved it. I have 7.3 from ondrej ------------------------------------------------------------------------ [2019-05-07 12:37:01] victoria at sharksmedia dot dk Here goes: (gdb) p meta $1 = (MYSQLND_FIELD *) 0x7f4929401018 (gdb) p *meta Cannot access memory at address 0x7f4929401018 (gdb) p *meta->sname Cannot access memory at address 0x7f4929401018 (gdb) disassemble Dump of assembler code for function mysqlnd_mysqlnd_res_meta_free_pub: 0x00007f49392c73c0 <+0>: push %r12 0x00007f49392c73c2 <+2>: push %rbp 0x00007f49392c73c3 <+3>: push %rbx 0x00007f49392c73c4 <+4>: mov (%rdi),%rbx 0x00007f49392c73c7 <+7>: test %rbx,%rbx 0x00007f49392c73ca <+10>: je 0x7f49392c7424 <mysqlnd_mysqlnd_res_meta_free_pub+100> 0x00007f49392c73cc <+12>: mov 0x14(%rdi),%eax 0x00007f49392c73cf <+15>: mov %rdi,%r12 0x00007f49392c73d2 <+18>: lea (%rax,%rax,4),%rbp 0x00007f49392c73d6 <+22>: shl $0x5,%rbp 0x00007f49392c73da <+26>: add %rbx,%rbp 0x00007f49392c73dd <+29>: jmp 0x7f49392c7417 <mysqlnd_mysqlnd_res_meta_free_pub+87> 0x00007f49392c73df <+31>: nop 0x00007f49392c73e0 <+32>: test %rbx,%rbx 0x00007f49392c73e3 <+35>: je 0x7f49392c7410 <mysqlnd_mysqlnd_res_meta_free_pub+80> => 0x00007f49392c73e5 <+37>: mov (%rbx),%rdi 0x00007f49392c73e8 <+40>: movq $0x0,0x90(%rbx) 0x00007f49392c73f3 <+51>: movq $0x0,0x48(%rbx) 0x00007f49392c73fb <+59>: test %rdi,%rdi 0x00007f49392c73fe <+62>: je 0x7f49392c7410 <mysqlnd_mysqlnd_res_meta_free_pub+80> 0x00007f49392c7400 <+64>: testb $0x40,0x4(%rdi) 0x00007f49392c7404 <+68>: jne 0x7f49392c7410 <mysqlnd_mysqlnd_res_meta_free_pub+80> 0x00007f49392c7406 <+70>: subl $0x1,(%rdi) 0x00007f49392c7409 <+73>: jne 0x7f49392c7410 <mysqlnd_mysqlnd_res_meta_free_pub+80> 0x00007f49392c740b <+75>: callq 0x7f49392b3340 <_efree@plt> 0x00007f49392c7410 <+80>: add $0xa0,%rbx 0x00007f49392c7417 <+87>: cmp %rbp,%rbx 0x00007f49392c741a <+90>: jne 0x7f49392c73e0 <mysqlnd_mysqlnd_res_meta_free_pub+32> 0x00007f49392c741c <+92>: movq $0x0,(%r12) 0x00007f49392c7424 <+100>: pop %rbx 0x00007f49392c7425 <+101>: pop %rbp 0x00007f49392c7426 <+102>: pop %r12 0x00007f49392c7428 <+104>: retq End of assembler dump. ------------------------------------------------------------------------ [2019-05-07 12:30:58] dmitry@php.net https://github.com/php/php-src/commit/5eb1f92f31cafc48384f9096012f421b37f6d425 shouldn't be the reason of the crash. Victoria, can you provide more info from gdb and coredump. Please, execute the following commands and capture the output. (gdb) p meta (gdb) p *meta (gdb) p *meta->sname (gdb) disassemble ------------------------------------------------------------------------ The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at https://bugs.php.net/bug.php?id=77955 -- Edit this bug report at https://bugs.php.net/bug.php?id=77955&edit=1