Bug #77955 [Asn]: Random segmentation fault in mysqlnd from php-fpm

From: Date: Tue, 07 May 2019 13:14:31 +0000
Subject: Bug #77955 [Asn]: Random segmentation fault in mysqlnd from php-fpm
References: 1  Groups: php.bugs 
Request: Send a blank email to php-bugs+get-220739@lists.php.net to get a copy of this message
Edit report at https://bugs.php.net/bug.php?id=77955&edit=1

 ID:                 77955
 Updated by:         dmitry@php.net
 Reported by:        victoria at sharksmedia dot dk
 Summary:            Random segmentation fault in mysqlnd from php-fpm
 Status:             Assigned
 Type:               Bug
 Package:            PDO MySQL
 Operating System:   CentOS 7.6.1810
 PHP Version:        7.3.4
 Assigned To:        dmitry
 Block user comment: N
 Private report:     N

 New Comment:

Few more commands.

(gdb) up
(gdb) p *meta
(gdb) p i
(gdb) p meta.fields[0]
(gdb) p meta.fields[i-1]

Probably, "meta.fields" was somehow corrupted or used after free, but I can't guess
how this might happen just analysing sources. I need a way to reproduce this, to catch the problem.


Previous Comments:
------------------------------------------------------------------------
[2019-05-07 13:05:14] nikic@php.net

My theory here would be that we need to swap the free_result_buffer() and free_metadata() calls
inside https://github.com/php/php-src/blob/master/ext/mysqlnd/mysqlnd_result.c#L293,
because free_result_internal() will destroy the mempool, which is shared with the metadata.

Does that sound plausible Dmitry?

------------------------------------------------------------------------
[2019-05-07 12:52:02] Scott at exussum dot co dot uk

Not sure if it helps but we have a long running job which gets this same issue. It happens at a
different point each time.  My guess was it was something to do with gc but not proved it. 

I have 7.3 from ondrej

------------------------------------------------------------------------
[2019-05-07 12:37:01] victoria at sharksmedia dot dk

Here goes:

(gdb) p meta
$1 = (MYSQLND_FIELD *) 0x7f4929401018
(gdb) p *meta
Cannot access memory at address 0x7f4929401018
(gdb) p *meta->sname
Cannot access memory at address 0x7f4929401018
(gdb) disassemble
Dump of assembler code for function mysqlnd_mysqlnd_res_meta_free_pub:
   0x00007f49392c73c0 <+0>:	push   %r12
   0x00007f49392c73c2 <+2>:	push   %rbp
   0x00007f49392c73c3 <+3>:	push   %rbx
   0x00007f49392c73c4 <+4>:	mov    (%rdi),%rbx
   0x00007f49392c73c7 <+7>:	test   %rbx,%rbx
   0x00007f49392c73ca <+10>:	je     0x7f49392c7424
<mysqlnd_mysqlnd_res_meta_free_pub+100>
   0x00007f49392c73cc <+12>:	mov    0x14(%rdi),%eax
   0x00007f49392c73cf <+15>:	mov    %rdi,%r12
   0x00007f49392c73d2 <+18>:	lea    (%rax,%rax,4),%rbp
   0x00007f49392c73d6 <+22>:	shl    $0x5,%rbp
   0x00007f49392c73da <+26>:	add    %rbx,%rbp
   0x00007f49392c73dd <+29>:	jmp    0x7f49392c7417
<mysqlnd_mysqlnd_res_meta_free_pub+87>
   0x00007f49392c73df <+31>:	nop
   0x00007f49392c73e0 <+32>:	test   %rbx,%rbx
   0x00007f49392c73e3 <+35>:	je     0x7f49392c7410
<mysqlnd_mysqlnd_res_meta_free_pub+80>
=> 0x00007f49392c73e5 <+37>:	mov    (%rbx),%rdi
   0x00007f49392c73e8 <+40>:	movq   $0x0,0x90(%rbx)
   0x00007f49392c73f3 <+51>:	movq   $0x0,0x48(%rbx)
   0x00007f49392c73fb <+59>:	test   %rdi,%rdi
   0x00007f49392c73fe <+62>:	je     0x7f49392c7410
<mysqlnd_mysqlnd_res_meta_free_pub+80>
   0x00007f49392c7400 <+64>:	testb  $0x40,0x4(%rdi)
   0x00007f49392c7404 <+68>:	jne    0x7f49392c7410
<mysqlnd_mysqlnd_res_meta_free_pub+80>
   0x00007f49392c7406 <+70>:	subl   $0x1,(%rdi)
   0x00007f49392c7409 <+73>:	jne    0x7f49392c7410
<mysqlnd_mysqlnd_res_meta_free_pub+80>
   0x00007f49392c740b <+75>:	callq  0x7f49392b3340 <_efree@plt>
   0x00007f49392c7410 <+80>:	add    $0xa0,%rbx
   0x00007f49392c7417 <+87>:	cmp    %rbp,%rbx
   0x00007f49392c741a <+90>:	jne    0x7f49392c73e0
<mysqlnd_mysqlnd_res_meta_free_pub+32>
   0x00007f49392c741c <+92>:	movq   $0x0,(%r12)
   0x00007f49392c7424 <+100>:	pop    %rbx
   0x00007f49392c7425 <+101>:	pop    %rbp
   0x00007f49392c7426 <+102>:	pop    %r12
   0x00007f49392c7428 <+104>:	retq
End of assembler dump.

------------------------------------------------------------------------
[2019-05-07 12:30:58] dmitry@php.net

https://github.com/php/php-src/commit/5eb1f92f31cafc48384f9096012f421b37f6d425
shouldn't be the reason of the crash.

Victoria, can you provide more info from gdb and coredump.
Please, execute the following commands and capture the output.

(gdb) p meta
(gdb) p *meta
(gdb) p *meta->sname
(gdb) disassemble

------------------------------------------------------------------------
[2019-05-07 10:09:46] victoria at sharksmedia dot dk

Thanks for the suggestion,

For your information; i have been trying to get Remi's package setup working on my server in
order recompile and test this, but i have not had much success building php-mysqlnd yet, i'm
currently waiting for a reply from Remi.

If you happen to know Remi's setup, or can provide a setup that allows me to compile a
identical or near identical mysqlnd.so binary i'm happy to try it out.

I could compile vanilla php from scratch, but i'm afraid to introduce new problems.

------------------------------------------------------------------------


The remainder of the comments for this report are too long. To view
the rest of the comments, please view the bug report online at

    https://bugs.php.net/bug.php?id=77955


--
Edit this bug report at https://bugs.php.net/bug.php?id=77955&edit=1


Thread (25 messages)

« previous php.bugs (#220739) next »