Bug #76436 [Opn->Fbk]: mysqli memory leak mysqli_fetch_object

Edit report at https://bugs.php.net/bug.php?id=76436&edit=1 ID: 76436 Updated by: cmb@php.net Reported by: luca dot looz92 at gmail dot com Summary: mysqli memory leak mysqli_fetch_object -Status: Open +Status: Feedback Type: Bug Package: MySQLi related Operating System: centos 7 PHP Version: 7.2.6 -Assigned To: +Assigned To: cmb Block user comment: N Private report: N New Comment: > On master/7.3 this leak is no longer present […] So this ticket would be obsolete, since 7.3 is the oldest version still receiving active support. Previous Comments: ------------------------------------------------------------------------ [2019-08-14 21:50:05] phpbug at ethaniel dot com I have this problem in PHP 5.6.40 on Centos 7.6. This simple code triggers it. I just read around 1 million rows from the table and my memory usage is just growing higher and higher. $cnt = 0; $result = mysqli_query($conn,"SELECT text,sms_date,to FROM sms_data.sms_201933;"); while ($row = mysqli_fetch_object($result)) { if ($cnt%1000) { echo memory_get_usage()." *** \n\n"; } $cnt++; } ------------------------------------------------------------------------ [2018-06-16 19:01:38] luca dot looz92 at gmail dot com I think that i have found the real issue: persistent string references inside zval array that are forgotten because in fast_shutdown mode symbols aren't destroyed and instead the shutdown executor trusts the ZMM for releasing the entire memory directly. On the 7.2 branch mysqlnd_wireprotocol.c allocates meta field string names in persistent memory. I don't know in which case is best to use direct memory or ZMM anyhow these strings are correctly released in mysqlnd_result_meta.c. The "leak" happens inside mysqlnd_result.c when fetching with MYSQLND_FETCH_ASSOC, the field name is added as a key to the result array through zend_hash_update which internally calls zend_string_addref to keep a strong reference. Normally all of this shouldn't be an issue but with the fast shutdown mode which doesn't releases each symbol one by one then becomes a leak because the array isn't released through the normal flow and neither the associative key with zend_string_release. Obviously if you allocate the meta field name through ZMM then the leak doesn't happen anymore. On 7.1 / 7.0 the leak isn't present because the integrated fast shutdown mode was introduced with 7.2. On master/7.3 this leak is no longer present because of some refactoring made on mysqlnd which now uses interned strings allocated with ZMM. Note that this can still happen on master if someone adds strong references inside zval symbols to memory not allocated through ZMM. I don't know if the real solution should be to do some refactoring regarding the fast shutdown mode or if simply who allocates persistent memory must be extra cautious to avoid this to happen. In the latter case probably is better to backport some of the refactoring done on master in mysqlnd. ------------------------------------------------------------------------ [2018-06-09 21:24:10] luca dot looz92 at gmail dot com I've discovered that in debug mode the leak doesn't happen because "fast_shutdown" of "shutdown_executor" in "Zend/zend_execute_API.c" is 0. In particular seems that the call zend_objects_store_free_object_storage(&EG(objects_store), fast_shutdown); is the culprit. If I force fast_shutdown at 0 only for that line then even in release mode the leak doesn't happen anymore (tested also with php-fpm to be sure). I don't know if at this point the leak is in the internal memory management or inside the mysqli/mysqlnd extensions. I've also tracked the calls of "zend_string_init"/"zend_string_release" of the mysql result meta->sname and apparently the calls count matches... ------------------------------------------------------------------------ [2018-06-09 20:36:23] luca dot looz92 at gmail dot com The leak doesn't happen on 7.3.0alpha1 just compiled from the source. In addition to valgrind i'm also running the sample php script under php-fpm and sending requests with "vegeta" for a couple of hours. ------------------------------------------------------------------------ [2018-06-09 19:51:37] luca dot looz92 at gmail dot com After digging a bit i think that maybe the trace reported by valgrind is incorrect and reports some other memory. The leak is definitively present but when it happens it's about 4 byte for an entire request. The leak isn't present on 7.1.x ------------------------------------------------------------------------ The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at https://bugs.php.net/bug.php?id=76436 -- Edit this bug report at https://bugs.php.net/bug.php?id=76436&edit=1