Bug #76436 [Opn]: mysqli memory leak mysqli_fetch_object

Edit report at https://bugs.php.net/bug.php?id=76436&edit=1 ID: 76436 User updated by: luca dot looz92 at gmail dot com Reported by: luca dot looz92 at gmail dot com Summary: mysqli memory leak mysqli_fetch_object Status: Open Type: Bug Package: MySQLi related Operating System: centos 7 PHP Version: 7.2.6 Block user comment: N Private report: N New Comment: I think that i have found the real issue: persistent string references inside zval array that are forgotten because in fast_shutdown mode symbols aren't destroyed and instead the shutdown executor trusts the ZMM for releasing the entire memory directly. On the 7.2 branch mysqlnd_wireprotocol.c allocates meta field string names in persistent memory. I don't know in which case is best to use direct memory or ZMM anyhow these strings are correctly released in mysqlnd_result_meta.c. The "leak" happens inside mysqlnd_result.c when fetching with MYSQLND_FETCH_ASSOC, the field name is added as a key to the result array through zend_hash_update which internally calls zend_string_addref to keep a strong reference. Normally all of this shouldn't be an issue but with the fast shutdown mode which doesn't releases each symbol one by one then becomes a leak because the array isn't released through the normal flow and neither the associative key with zend_string_release. Obviously if you allocate the meta field name through ZMM then the leak doesn't happen anymore. On 7.1 / 7.0 the leak isn't present because the integrated fast shutdown mode was introduced with 7.2. On master/7.3 this leak is no longer present because of some refactoring made on mysqlnd which now uses interned strings allocated with ZMM. Note that this can still happen on master if someone adds strong references inside zval symbols to memory not allocated through ZMM. I don't know if the real solution should be to do some refactoring regarding the fast shutdown mode or if simply who allocates persistent memory must be extra cautious to avoid this to happen. In the latter case probably is better to backport some of the refactoring done on master in mysqlnd. Previous Comments: ------------------------------------------------------------------------ [2018-06-09 21:24:10] luca dot looz92 at gmail dot com I've discovered that in debug mode the leak doesn't happen because "fast_shutdown" of "shutdown_executor" in "Zend/zend_execute_API.c" is 0. In particular seems that the call zend_objects_store_free_object_storage(&EG(objects_store), fast_shutdown); is the culprit. If I force fast_shutdown at 0 only for that line then even in release mode the leak doesn't happen anymore (tested also with php-fpm to be sure). I don't know if at this point the leak is in the internal memory management or inside the mysqli/mysqlnd extensions. I've also tracked the calls of "zend_string_init"/"zend_string_release" of the mysql result meta->sname and apparently the calls count matches... ------------------------------------------------------------------------ [2018-06-09 20:36:23] luca dot looz92 at gmail dot com The leak doesn't happen on 7.3.0alpha1 just compiled from the source. In addition to valgrind i'm also running the sample php script under php-fpm and sending requests with "vegeta" for a couple of hours. ------------------------------------------------------------------------ [2018-06-09 19:51:37] luca dot looz92 at gmail dot com After digging a bit i think that maybe the trace reported by valgrind is incorrect and reports some other memory. The leak is definitively present but when it happens it's about 4 byte for an entire request. The leak isn't present on 7.1.x ------------------------------------------------------------------------ [2018-06-09 09:57:33] luca dot looz92 at gmail dot com Description: ------------ I was going crazy by investigating a memory issue on our server, php-fpm pools after a couple of days exhausts 16gb of RAM. By doing a lot of tests with trials and errors i was able to reproduce the issue. The memory leak happens when using functions like "mysqli_fetch_object" and "mysqli_fetch_array". Doesn't happen with "mysqli_fetch_row". I'm able to reproduce this by compiling php 7.2.6 from source with just the mysqli module: ./configure --disable-all --with-mysqli A very strange thing is that if I compile php in debug mode the leak doesn't happen anymore... ./configure --disable-all --enable-debug --with-mysqli If I put the code in a loop the leak doesn't become bigger so it seems that is limited per script execution by leaking something like 4 bytes for each execution. To test this I used the Wordpress default database by reading the hello world article. I have attached the valgrind results with "export ZEND_DONT_UNLOAD_MODULES=1". Another thing to note is that if set "export USE_ZEND_ALLOC=0" with the non-debug version the leak doesn't happen. I've seen this issue on 7.2.2, 7.2.5 and 7.2.6. Test script: --------------- <?php $dbh = mysqli_init(); mysqli_real_connect( $dbh, '127.0.0.1', 'root', 'mydbpass'); mysqli_select_db( $dbh, 'dbwpempty' ); $result = mysqli_query( $dbh, "SELECT * FROM wp_posts WHERE ID = 1 LIMIT 1" ); $num_rows = 0; $last_result = array(); while ( $row = mysqli_fetch_object( $result ) ) { $last_result[$num_rows] = $row; $num_rows++; } $result->free(); $dbh->close(); ?> valgrind --leak-check=full php mysqlleak.php Expected result: ---------------- ==30485== HEAP SUMMARY: ==30485== in use at exit: 32 bytes in 1 blocks ==30485== total heap usage: 10,515 allocs, 10,514 frees, 1,983,262 bytes allocated ==30485== ==30485== LEAK SUMMARY: ==30485== definitely lost: 0 bytes in 0 blocks ==30485== indirectly lost: 0 bytes in 0 blocks ==30485== possibly lost: 0 bytes in 0 blocks ==30485== still reachable: 32 bytes in 1 blocks ==30485== suppressed: 0 bytes in 0 blocks Actual result: -------------- ==30483== HEAP SUMMARY: ==30483== in use at exit: 936 bytes in 24 blocks ==30483== total heap usage: 9,275 allocs, 9,251 frees, 1,466,918 bytes allocated ==30483== ==30483== 904 bytes in 23 blocks are definitely lost in loss record 2 of 2 ==30483== at 0x4C29C23: malloc (vg_replace_malloc.c:299) ==30483== by 0x5B1198: __zend_malloc (zend_alloc.c:2829) ==30483== by 0x564684: zend_string_alloc (zend_string.h:134) ==30483== by 0x564684: zend_string_init (zend_string.h:170) ==30483== by 0x564684: php_mysqlnd_rset_field_read (mysqlnd_wireprotocol.c:1378) ==30483== by 0x56DA81: mysqlnd_mysqlnd_res_meta_read_metadata_pub (mysqlnd_result_meta.c:76) ==30483== by 0x56A2B6: mysqlnd_mysqlnd_res_read_result_metadata_pub (mysqlnd_result.c:385) ==30483== by 0x56C8D0: mysqlnd_query_read_result_set_header (mysqlnd_result.c:539) ==30483== by 0x55B152: mysqlnd_mysqlnd_conn_data_reap_query_pub (mysqlnd_connection.c:917) ==30483== by 0x55D79E: mysqlnd_mysqlnd_conn_data_query_pub (mysqlnd_connection.c:859) ==30483== by 0x4AA001: zif_mysqli_query (mysqli_nonapi.c:593) ==30483== by 0x671C93: ZEND_DO_ICALL_SPEC_RETVAL_USED_HANDLER (zend_vm_execute.h:617) ==30483== by 0x671C93: execute_ex (zend_vm_execute.h:59734) ==30483== by 0x67A690: zend_execute (zend_vm_execute.h:63760) ==30483== by 0x5D8FC7: zend_execute_scripts (zend.c:1496) ==30483== ==30483== LEAK SUMMARY: ==30483== definitely lost: 904 bytes in 23 blocks ==30483== indirectly lost: 0 bytes in 0 blocks ==30483== possibly lost: 0 bytes in 0 blocks ==30483== still reachable: 32 bytes in 1 blocks ==30483== suppressed: 0 bytes in 0 blocks ==30483== Reachable blocks (those to which a pointer was found) are not shown. ------------------------------------------------------------------------ -- Edit this bug report at https://bugs.php.net/bug.php?id=76436&edit=1