Bug #68318 [Opn->Nab]: mysqlnd Memory allocation problem - memory not freed

Edit report at https://bugs.php.net/bug.php?id=68318&edit=1 ID: 68318 Updated by: rasmus@php.net Reported by: tomas at slax dot org Summary: mysqlnd Memory allocation problem - memory not freed -Status: Open +Status: Not a bug Type: Bug Package: MySQLi related Operating System: Linux PHP Version: 5.5.18 Block user comment: N Private report: N New Comment: The huge piece you are missing is that libmysqlclient doesn't use the PHP memory manager so any memory it uses is not counted when you call memory_get_usage(). So it makes perfect sense that your libmysqlclient example stays constant. It could be leaking Gigabytes of memory and you would still see constant numbers from memory_get_usage() so using that as any sort of proof of a problem is invalid. Looking at your script itself, it looks ok to me. When you fetch a row from the server into the client, the client allocates memory for that row. libmysqlclient does the same, just using malloc() not emalloc(). I bet actual memory usage is identical if you looked at it with something like Massif. The memory should eventually get free'ed. And there are some recent optimizations here too. Have a read through http://blog.ulf-wendel.de/2014/php-5-7-mysqlnd-memory-optimizations/ Previous Comments: ------------------------------------------------------------------------ [2014-10-28 12:48:31] tomas at slax dot org Description: ------------ When PHP is compiled with mysqlnd (default), it has a problem with freeing memory, as I will demonstrate in test script. If PHP is compiled with libmysqlclient, the problem does not occur. Compiling PHP with mysqlnd causes the test script to work INCORRECTLY: ./configure \ --with-mysql-sock=/var/run/mysql/mysql.sock \ --with-mysql \ --with-mysqli # note that mysqlnd is enabled by default. It's like: --with-mysqli=mysqlnd Compile PHP with libmysqlclient to fix the memory allocation problem: ./configure \ --with-mysql-sock=/var/run/mysql/mysql.sock \ --with-mysql=/usr \ --with-mysqli=/usr/bin/mysql_config \ # note that this fixes the problem I expect you to fix bug in PHP's mysqlnd driver so it properly free()s the memory!! Thank you very much !!! This error is there since PHP 5.4.x and is there even now with PHP 5.6.x Test script: --------------- <?php echo "First, connect to database test...\n"; $link=mysqli_connect("localhost",'root','rrr','test'); echo "Create temporary table with lots of rows...\n"; $ok=mysqli_query($link,"CREATE TEMPORARY TABLE tbl123 LIKE mysql.user"); if (!$ok) die(); for ($i=0;$i<6000; $i++) { mysqli_query($link,"INSERT IGNORE INTO tbl123 SET host='x".$i."'"); } echo "Get all rows from database\n"; $result=mysqli_query($link,"SELECT * FROM tbl123"); while($row=mysqli_fetch_row($result)) { echo "Mem used: ".number_format(memory_get_usage()/1024,1,'.','')." KB\n"; } // expected result: memory usage still the same // libmysqlclient result: memory usage still the same // mysqlnd result: memory usage increases with each row!! :( // and if it reaches php memory limit, the script dies :( ?> Expected result: ---------------- I expect that PHP doesn't use more than, say, 512KB while processing 6000 rows from database as shown in the example. I expect the test script to print constant memory usage, as like: Mem used: 228.7 KB Mem used: 228.7 KB Mem used: 228.7 KB Mem used: 228.7 KB Mem used: 228.7 KB Mem used: 228.7 KB Mem used: 228.7 KB Mem used: 228.7 KB Mem used: 228.7 KB (still the same) Actual result: -------------- When using mysqlnd (compiled with mysqlnd by default), it increases memory usage with every row, this is very bad: Mem used: 14040.2 KB Mem used: 14042.1 KB Mem used: 14044.1 KB Mem used: 14046.1 KB Mem used: 14048.0 KB Mem used: 14050.0 KB Mem used: 14052.0 KB Mem used: 14054.0 KB Mem used: 14055.9 KB Mem used: 14057.9 KB Mem used: 14059.9 KB Mem used: 14061.8 KB Mem used: 14063.8 KB Mem used: 14065.8 KB (still bigger, this is WRONG) ------------------------------------------------------------------------ -- Edit this bug report at https://bugs.php.net/bug.php?id=68318&edit=1