Bug #68318 [Nab]: mysqlnd Memory allocation problem - memory not freed
Edit report at https://bugs.php.net/bug.php?id=68318&edit=1
ID: 68318
User updated by: tomas at slax dot org
Reported by: tomas at slax dot org
Summary: mysqlnd Memory allocation problem - memory not freed
Status: Not a bug
Type: Bug
Package: MySQLi related
Operating System: Linux
PHP Version: 5.5.18
Block user comment: N
Private report: N
New Comment:
Hi, I noticed you changed this to "not a bug". What a shame.
This is definitely a bug.
The memory DOES NOT get free'ed eventually. It NEVER gets free'ed and the PHP script DIES
on memory limit! I am sorry for using exclamation marks here, but you're probably completely
missing the point.
My primary problem was that a script similar to this one (with a simple while loop) died
unexpectedly on memory limit for no reason, since the $row variable is reused million times, there
is no reason for it to remain in memory.
My concern is not that used memory reported by PHP's function memory_get_usage() is increasing.
I don't care about that reported value, really. What I care about is that if my script outputs
more rows (like a million rows) in a simple while loop, the memory consumed (and NEVER free'ed)
increases to the point that the script dies on reaching the max available memory limit (configured
in php.ini). There is something really wrong with that, I see no reason whatsoever why PHP should
allocate memory and never free it.
I am ready to provide more information in order to convince you that this is a real problem in PHP.
Just let me know please what do you need from me. Thank you very much.
Previous Comments:
------------------------------------------------------------------------
[2014-10-29 05:56:41] rasmus@php.net
The huge piece you are missing is that libmysqlclient doesn't use the PHP memory manager so any
memory it uses is not counted when you call memory_get_usage(). So it makes perfect sense that your
libmysqlclient example stays constant. It could be leaking Gigabytes of memory and you would still
see constant numbers from memory_get_usage() so using that as any sort of proof of a problem is
invalid.
Looking at your script itself, it looks ok to me. When you fetch a row from the server into the
client, the client allocates memory for that row. libmysqlclient does the same, just using malloc()
not emalloc(). I bet actual memory usage is identical if you looked at it with something like
Massif. The memory should eventually get free'ed. And there are some recent optimizations here
too. Have a read through http://blog.ulf-wendel.de/2014/php-5-7-mysqlnd-memory-optimizations/
------------------------------------------------------------------------
[2014-10-28 12:48:31] tomas at slax dot org
Description:
------------
When PHP is compiled with mysqlnd (default), it has a problem with freeing memory, as I will
demonstrate in test script. If PHP is compiled with libmysqlclient, the problem does not occur.
Compiling PHP with mysqlnd causes the test script to work INCORRECTLY:
./configure \
--with-mysql-sock=/var/run/mysql/mysql.sock \
--with-mysql \
--with-mysqli
# note that mysqlnd is enabled by default. It's like: --with-mysqli=mysqlnd
Compile PHP with libmysqlclient to fix the memory allocation problem:
./configure \
--with-mysql-sock=/var/run/mysql/mysql.sock \
--with-mysql=/usr \
--with-mysqli=/usr/bin/mysql_config \
# note that this fixes the problem
I expect you to fix bug in PHP's mysqlnd driver so it properly free()s the memory!!
Thank you very much !!!
This error is there since PHP 5.4.x and is there even now with PHP 5.6.x
Test script:
---------------
<?php
echo "First, connect to database test...\n";
$link=mysqli_connect("localhost",'root','rrr','test');
echo "Create temporary table with lots of rows...\n";
$ok=mysqli_query($link,"CREATE TEMPORARY TABLE tbl123 LIKE mysql.user");
if (!$ok) die();
for ($i=0;$i<6000; $i++)
{
mysqli_query($link,"INSERT IGNORE INTO tbl123 SET
host='x".$i."'");
}
echo "Get all rows from database\n";
$result=mysqli_query($link,"SELECT * FROM tbl123");
while($row=mysqli_fetch_row($result))
{
echo "Mem used:
".number_format(memory_get_usage()/1024,1,'.','')." KB\n";
}
// expected result: memory usage still the same
// libmysqlclient result: memory usage still the same
// mysqlnd result: memory usage increases with each row!! :(
// and if it reaches php memory limit, the script dies :(
?>
Expected result:
----------------
I expect that PHP doesn't use more than, say, 512KB while processing 6000 rows from database as
shown in the example. I expect the test script to print constant memory usage, as like:
Mem used: 228.7 KB
Mem used: 228.7 KB
Mem used: 228.7 KB
Mem used: 228.7 KB
Mem used: 228.7 KB
Mem used: 228.7 KB
Mem used: 228.7 KB
Mem used: 228.7 KB
Mem used: 228.7 KB
(still the same)
Actual result:
--------------
When using mysqlnd (compiled with mysqlnd by default), it increases memory usage with every row,
this is very bad:
Mem used: 14040.2 KB
Mem used: 14042.1 KB
Mem used: 14044.1 KB
Mem used: 14046.1 KB
Mem used: 14048.0 KB
Mem used: 14050.0 KB
Mem used: 14052.0 KB
Mem used: 14054.0 KB
Mem used: 14055.9 KB
Mem used: 14057.9 KB
Mem used: 14059.9 KB
Mem used: 14061.8 KB
Mem used: 14063.8 KB
Mem used: 14065.8 KB
(still bigger, this is WRONG)
------------------------------------------------------------------------
--
Edit this bug report at https://bugs.php.net/bug.php?id=68318&edit=1
Thread (6 messages)