Edit report at https://bugs.php.net/bug.php?id=77308&edit=1
ID: 77308
User updated by: sjon at hortensius dot net
Reported by: sjon at hortensius dot net
Summary: Unbuffered queries memory leak
Status: Assigned
Type: Bug
Package: MySQL related
Operating System: archlinux
PHP Version: 7.3.0
Assigned To: dmitry
Block user comment: N
Private report: N
New Comment:
I've already bisected this as mentioned, caused by f1f1f6
Previous Comments:
------------------------------------------------------------------------
[2019-01-02 11:14:56] nikic@php.net
@dmity: Probably related to the mysqlnd arena improvements.
Massif trace:
97.88% (10,719,359B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->91.19% (9,987,510B) 0x953317: __zend_malloc (zend_alloc.c:2904)
| ->76.79% (8,409,934B) 0x9525C8: _emalloc (zend_alloc.c:2494)
| | ->73.05% (8,000,688B) 0x89116A: _mysqlnd_emalloc (mysqlnd_alloc.c:99)
| | | ->72.90% (7,984,000B) 0x8D3CDD: mysqlnd_arena_alloc (mysqlnd_block_alloc.c:66)
| | | | ->72.90% (7,984,000B) 0x8D430D: mysqlnd_mempool_get_chunk (mysqlnd_block_alloc.c:148)
| | | | ->72.90% (7,984,000B) 0x8A119A: php_mysqlnd_read_row_ex (mysqlnd_wireprotocol.c:1384)
| | | | ->72.90% (7,984,000B) 0x8A358B: php_mysqlnd_rowp_read (mysqlnd_wireprotocol.c:1724)
| | | | ->72.90% (7,984,000B) 0x8BF0CE: mysqlnd_mysqlnd_result_unbuffered_fetch_row_c_pub
(mysqlnd_result.c:677)
| | | | ->72.90% (7,984,000B) 0x8C5755: mysqlnd_mysqlnd_res_fetch_row_c_pub
(mysqlnd_result.c:1757)
| | | | ->72.90% (7,984,000B) 0x65C177: pdo_mysql_stmt_fetch (mysql_statement.c:660)
| | | | ->72.90% (7,984,000B) 0x64853F: do_fetch_common (pdo_stmt.c:676)
| | | | ->72.90% (7,984,000B) 0x648BF1: do_fetch (pdo_stmt.c:831)
| | | | ->72.90% (7,984,000B) 0x64F1BF: pdo_stmt_iter_move_forwards
(pdo_stmt.c:2447)
| | | | ->72.90% (7,984,000B) 0xA214AA: ZEND_FE_FETCH_R_SPEC_VAR_HANDLER
(zend_vm_execute.h:21484)
| | | | ->72.90% (7,984,000B) 0xA624CD: execute_ex (zend_vm_execute.h:57707)
| | | | ->72.90% (7,984,000B) 0xA656DA: zend_execute
(zend_vm_execute.h:60833)
| | | | ->72.90% (7,984,000B) 0x98AE19: zend_execute_scripts
(zend.c:1568)
| | | | ->72.90% (7,984,000B) 0x8F0BE1: php_execute_script
(main.c:2630)
| | | | ->72.90% (7,984,000B) 0xA6842F: do_cli (php_cli.c:997)
| | | | ->72.90% (7,984,000B) 0xA695A5: main (php_cli.c:1389)
Presumably freeing chunks from the arena doesn't work properly anymore.
------------------------------------------------------------------------
[2019-01-02 10:58:17] nikic@php.net
Test script including (slow) table generation:
<?php
ini_set('memory_limit', '8M');
$pdo = new PDO('mysql:host=127.0.0.1;dbname=test', 'php-test',
'password');
$pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$pdo->setAttribute(PDO::MYSQL_ATTR_USE_BUFFERED_QUERY, false);
$pdo->exec("DROP TABLE IF EXISTS large_table");
$pdo->exec("CREATE TABLE large_table (id int)");
for ($j = 0; $j < 100; $j++) {
$query = "INSERT INTO large_table (id) VALUES ";
for ($i = 0; $i < 10000; $i++) {
$query .= "($i), ";
}
$query .= "($i)";
$pdo->exec($query);
}
$result = $pdo->query("SELECT * FROM large_table");
foreach ($result as $i => $row)
if ($i % 10000 === 0)
printf("%10s %10s\n", $i, memory_get_usage());
I'm also getting a
Warning: Packets out of order. Expected 231 received 48. Packet size=3422212 in Unknown on line 0
after the memory limit error.
------------------------------------------------------------------------
[2018-12-28 21:06:31] cweiske@php.net
This breaks Roundcube webmail, see https://bugs.archlinux.org/task/61139.
Happens here with 7.3.0-2 (Debian 9)
------------------------------------------------------------------------
[2018-12-17 09:16:01] sjon at hortensius dot net
Description:
------------
commit f1f1f6 broke unbuffered queries in mysqlnd; there is a memory leak causing php to run out of
memory
Test script:
---------------
<?php
ini_set('memory_limit', '8M');
$pdo = new PDO('mysql:host=*;dbname=*', '*', '*');
$pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$pdo->setAttribute(PDO::MYSQL_ATTR_USE_BUFFERED_QUERY, false);
$result = $pdo->query("SELECT * FROM large_table");
foreach ($result as $i => $row)
if ($i % 10000 === 0)
printf("%10s %10s\n", $i, memory_get_usage());
Expected result:
----------------
reasonable constant memory usage, eg:
0 427648
10000 427936
20000 427968
30000 427968
40000 427936
Actual result:
--------------
0 427296
10000 2000288
20000 3622336
30000 5260736
40000 6915488
Fatal error: Allowed memory size of 8388608 bytes exhausted (tried to allocate 16384 bytes) in /php
on line 10
------------------------------------------------------------------------
--
Edit this bug report at https://bugs.php.net/bug.php?id=77308&edit=1