#40261 [Com]: Extremely slow data handling due to memory fragmentation

ID: 40261 Comment by: bloire at citytoo dot com Reported By: thuejk at gmail dot com Status: Assigned Bug Type: Performance problem Operating System: All PHP Version: 5.2.0 Assigned To: dmitry New Comment: My version is 5.2.1 the lastest with mandake limited edition 2005 and fedora core 4 The probleme exist always with php 5.2.1 !!!!!!!!!!! The latest release ! Probleme looks like : function test(&$b){ //sql Query... //$a_result_mysql //No problem with performance with php 5.2.1 (6 secondes) foreach($a_result_mysql as $i=> $a_row) { $b[artist][item][$i] = $a_row; } //Very very very very very very very very very slow with php 5.2 (180 secondes) //normal with php < 5.2 (10 secondes) foreach($a_result_mysql as $i=> $a_row) { $b[artist][item][$i][name] = $a_row[name]; $b[artist][item][$i][firstname] = $a_row[firstname]; $b[artist][item][$i][address] = $a_row[address]; $b[artist][item][$i][zip] = $a_row[name]; $b[artist][item][$i][color] = $a_row[color]; } } test($b); Previous Comments: ------------------------------------------------------------------------ [2007-03-16 13:02:56] bloire at citytoo dot com I have exactly the same probleme with foreach with big rows from mysql and reaffection for each field for each row After that, all processing is verry verry verry VERRY slow. It's HORRIBLE. It's verry strange because I didn"t have this probleme with php < 5.2 I hope that will be repair because now, I'm not trusting with php 5.2 Thank you for all ------------------------------------------------------------------------ [2007-03-16 12:09:04] thuejk at gmail dot com I think I am hitting this in practice, or something like it. I have a function call <? function lala() { [database access with lots of data] echo "returning " . time(); return; } lala(); echo "returned ".time(); ?> And I can see that for some reason, the time between "returning" and "returned" is 60 seconds! This only happens the first time this function is called, for some reason. Installing php 5.1.6 it returns instantaneously. I liked thet PHP 5.1 memory allocator better :(. The PHP 5.1 memory allocator was also 1/4 the size of the PHP 5.2 memory allocator. ------------------------------------------------------------------------ [2007-01-30 14:53:40] thuejk at gmail dot com The latest PHP snapshot does fix my example, and probably makes my production code work. This will probably fix the problem in far most cases. But it is possible to construct an example which still have the problematic behavior. One example is below. <?php $num = 100000; $a = Array(); for ($i=0; $i<$num; $i++) { $a[$i] = Array(1); } for ($i=0; $i<$num; $i++) { $b[$i] = $a[$i][0]; } unset($a); for ($i=0; $i<$num; $i++) { $b[$i] = "1234567890123456789012345678901234567890123456789012345678901234567\ 8901234567890123456789012345678901234567890123456789012345678901234567890123456\ 7890123456789012345678901234567890123456789012345678901234567890123456789012345\ 6789012345678901234567890123456789012345678901234567890123456789012345678901234\ 5678901234567890"; } ?> ------------------------------------------------------------------------ [2007-01-30 14:03:32] dmitry@php.net Please try PHP_5_2 snapshot. It already uses litle bit different "best-fit" implementation and this script takes reasonable time (near the same as 5.1). ------------------------------------------------------------------------ [2007-01-29 13:24:47] thuejk at gmail dot com I added a few printf's and found out that the $num generated "holes" in the memory are 384 big. Zend has a special system for small allocations, but that only works for holes below ZEND_MM_SMALL_SIZE=280. The $num allocations at the end, which cause the problems, and 40 or 88 long. Note that these numbers are from a 64-bit machine, which of course has 8-byte pointers, and so a larger MM overhead. (The bug does also occur on 32bit machines) One temporary solution could be to raise #define ZEND_MM_NUM_BUCKETS 32 from which ZEND_MM_SMALL_SIZE is defined, so that ZEND_MM_SMALL_SIZE is made twice or perhaps 4 times as big. (I got a segfault when I tried that, haven't looked into why) A better solution would be to organize the free blocks in a balanced tree, instead of a linear list which has to be traversed. ------------------------------------------------------------------------ The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at http://bugs.php.net/40261 -- Edit this bug report at http://bugs.php.net/?id=40261&edit=1