Re: Re: Problem with memory management

Eric Butera wrote:

On Fri, Oct 10, 2008 at 3:32 PM, Nathan Rixham <nrixham@gmail.com> wrote:

Alan Boudreault wrote:

Hi all, I'm wondering why PHP doesn't free my memory with this test code. The memory usage is always the same even if i unset my object. The Garbage collector seems to only free the memory at the end of the script. Here's the php Scripts that i use for testing: <?php //dl("php_mapscript.so"); function test() { $oShapeFile = ms_newShapefileObj(

    "/opt/www/bdga/msp/data/bdga/BDGA_HYDRO_S_POLY.shp", -1);

echo "before getShape : ".memory_get_usage(true)."\n"; for ($i=0; $i<$oShapeFile->numshapes;$i++) {

    $oShape = $oShapeFile->getShape($i);
    $oShape->free();
    unset($oShape);
    $oShape = NULL;

} echo "after getShape : ".memory_get_usage(true)."\n"; $oShapeFile->free(); unset($oShapeFile); $oShapeFile = null;

echo "after free      : ".memory_get_usage(true)."\n";

} echo "start : ".memory_get_usage(true)."\n"; test(); echo "end : ".memory_get_usage(true)."\n"; ?> Output result: start : 262144 before getShape : 262144 after getShape : 11010048

after free      : 11010048

end : 11010048 I've also run valgrind to be sure that is not my extension that doesn't free its memory: $ valgrind --leak-check=full php -f shapeTest.php ==18730== LEAK SUMMARY:

==18730==    definitely lost: 0 bytes in 0 blocks.
==18730==      possibly lost: 0 bytes in 0 blocks.
==18730==    still reachable: 240 bytes in 2 blocks.
==18730==         suppressed: 0 bytes in 0 blocks.

Thanks, Alan

interesting, I'm finding the same thing in one of my atom feed parsers; over time no matter how much I unset / truncate variables the memory usage stills grows over time - at this time I can't find any way to bring it right down; on a timer the whole script resets and restarts itself; one thing I have noticed is that I can see the emalloc memory usage report dropping and rising [memory_get_usage( false );], but still rising over time.. hope this wasn't a hijack; just here trying to figure out the same problem at the minute! Regards.. Nathan -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php

I think the engine doesn't really worry about freeing memory until it has to. Or was this only on circular references? I remember there was that whole GSOC project to implement garbage collection, but it never got implemented. Maybe someone who really knows will chime in.

you're right; I remember the discussion over on internals 3rd December last year; they made a new gc patch for 5.3; not sure if it got implemented or not.. here's what andy wrote (there followed a massive discussion) [quote] Hi all, Was hoping to send this off earlier but I was travelling for the past week and had very limited email access. As promised in the past few weeks we have spent a significant amount of time in reviewing the garbage collector work and testing it in our performance lab. Dmitry has been exchanging ideas and patches with David Wang during this time. Suffice to say that as I've mentioned in the past, memory management is an extremely sensitive piece of PHP, which is why it was important for us to review this work in great depth before committing it to the code base. The updated patch still retains the same algorithm as David's original implementation however the data structures have been changed in order to work faster and use less memory. The revised patch has the following advantages: - It fixes several crash bugs - Enhances performance by removing several unnecessary checks - The memory overhead was reduced (from 12 to 4 bytes for each heap-allocated zval) - The speed of "clean" PHP code (code that doesn't create cycles) was improved - Additional test cases were created The end result is a more stable and faster GC patch. That said we have yet to find real-life applications that create significant cycles which would benefit from this patch. In fact as you'll see from the results our tests show an increase in maximum memory use and slower execution (to be fair they are marginal). We have tested both PHP_5_3 without any patches, the original patch and the new patch. The following table shows execution time (seconds for N requests) and slowdown. PHP_5_3 Original GC patch Current GC patch slowdown slowdown bench 11,550 12,310 6,58% 12,170 5,37% hello 8,781 8,852 0,81% 8,813 0,36% xoops 128,500 135,100 5,14% 130,200 1,32% static 18,540 20,840 12,41% 18,920 2,05% qdig 29,320 30,270 3,24% 29,610 0,99% qdig2 13,960 14,100 1,00% 14,090 0,93% The next table shows memory usage in MB and overhead PHP_5_3 Original GC patch Current GC patch overhead overhead hello 13,750 13,945 1,42% 13,765 0,11% xoops 18,036 18,636 3,33% 18,568 2,95% static 15,300 16,000 4,58% 15,308 0,05% qdig 14,820 15,008 1,27% 14,828 0,05% qdig2 14,824 15,012 1,27% 14,838 0,09% To summarize the patch lead to approx. 5% slowdown and 3% memory overhead for typical applications (as always, you mileage may vary depending on your system's architecture and OS although my guesstimate is that you will see even worse results in a 64bit environment). We also tested SugarCRM to get another sanity for these results and we got similar results. I am not quite sure where this leaves us with this patch. On one hand I think now the effort has been made it's a good thing to offer it as part of PHP. The downside is of course some performance degradation and possible instabilities as this patch continues to stabilize (it took about two releases for us to stabilize the Zend Memory Manager even after in-depth testing due to edge cases in allocation patterns and various extensions, etc...).I'd be surprised if our team has found all possible bugs. Personally I think the decision should be either in or out. Adding this as a compile option is not a good idea as it would create binary compatibility issues and would be a pain for the community. I think my inclination is to commit the patch, not have it #ifdef'ed but always compiled but to have the garbage collection itself turned off by default (mainly for stability reasons. Note: the increased memory footprint and performance loss also exists with the collection itself turned off). We can turn it on when we're in dev for snapshots so that we iron out bugs. That said, as you can see from the results most people and real-life applications will be worse off than today. Thanks to David & Dmitry for working hard on this (and everyone else who contributed). The stage is open for ideas/thoughts/suggestions J Andi [/quote]