Bug #64291 [Opn]: Indeterminate GC of evaled lambda function resources

Edit report at https://bugs.php.net/bug.php?id=64291&edit=1 ID: 64291 Updated by: dmitry@php.net Reported by: Terry at ellisons dot org dot uk Summary: Indeterminate GC of evaled lambda function resources Status: Open Type: Bug Package: Scripting Engine problem Operating System: Ubuntu 12.10 PHP Version: 5.4.12 Block user comment: N Private report: N New Comment: Your second script prints "a, a" only with OPCache, because it caches the included temporary file. Without OPCache it prints the expected "a, b". I also don't think that the first script indicates a bug. The more functions you create the more memory it requires. Previous Comments: ------------------------------------------------------------------------ [2013-11-22 12:34:08] Terry at ellisons dot org dot uk Just to note that my patch is not strong enough when used with OPcache, since a simple static running count can fail when the interpreter is forked. Better alternatives include * some form of unique id, e.g. a UUID or uniquid(true) * a content based hash, such as md5file(__FILE__) -- though if this were to be adopted then it would be better always to generate this as part of the compile creating say __FILE_MD5__, though this would add a few % to the compile time. This all needs wider discussion of these issues on the Internals ML. ------------------------------------------------------------------------ [2013-11-21 15:07:43] Terry at ellisons dot org dot uk The issue here is that the runtime defined function keys should be unique, and the current algo does not generate unique keys. There are many approaches that could be taken to remove such incorrect name clashes. The patch that I've submitted add a sequence count to the key -- simple but sufficient to prevent this bug. ------------------------------------------------------------------------ [2013-11-20 15:58:13] Terry at ellisons dot org dot uk Sorry the second line above should read: $b = function(){ return "b"; }; ------------------------------------------------------------------------ [2013-11-20 15:56:29] Terry at ellisons dot org dot uk This bug is cascading to another: https://bugs.php.net/bug.php?id=65915 -- consider this test script: <?php $tmp = '/tmp/testNg3hUy'; foreach (['a','b'] as $f) { file_put_contents($tmp, '<?php return function(){ return "'.$f.'"; };'); echo file_get_contents($tmp), "\n"; $$f = require $tmp; } printf( "%s, %s\n ", $a(), $b()); unlink($tmp); The require assignment should generate the equivalent of $a = function(){ return "a"; }; $a = function(){ return "b"; }; so that the printf should generate a, b However it actually generates a, a demonstrating that the "\0{closure}%s%p" naming convention does not generate unique closure names. ------------------------------------------------------------------------ [2013-02-24 14:24:05] Terry at ellisons dot org dot uk Description: ------------ The Internals thread "(non)growing memory while creating anoymous functions via eval()" see http://marc.info/?t=135990541000003&r=1&w=2 is the background to this report. Description Storage allocation and garbage collection of lambda function resources is indeterminate an may or may not lead to memory exhaustion. This (Example1) script demonstrates the effect: <?php $x = ""; while (1) { if (isset($argv[1])) $x = str_repeat(" " ,mt_rand(1,50000)); eval ("\$fun = function() { $x return memory_get_usage(); };"); echo "Mem usage= {$fun()}\n"; } If arg1 is set it dies with memory exhaustion, and is stable if unset. Replacing the eval with (giving Example2) $fun = function() { return memory_get_usage(); }; is also stable so this isn't a resource leakage in the string $x per se. The issue here is that the closure creates a magic name for the function, in PHP terms $function_name = sprintf("\0{closure}%s%p",name,addr); where the name is the resolved filename of the source where the function was defined and the addr is the absolute address of the function definition within the memory resident copy of the source during the compilation process, for example in Example 2 where $fun is statically defined, on my test this is "\0{closure}/tmp/y.php0x7fc90b1fd083" The compiler creates one entry for "\0{closure}/tmp/y.php0x7fc90b1fd08" in the CG function_table, but the closure DTOR does not delete or GC this entry. The reason is in this logical: in Example 2 the $fun assignment generates ZEND_DECLARE_LAMBDA_FUNCTION '\0{closure}/tmp/y.php0x7fc90b1fd083' ASSIGN !2, ~5 that is the function is compiled once but rebound multiple times during execution. With Example 1, the where the eval is used, the closure uses a name based on the source file and line number where the eval was executed, e.g. "/tmp/y.php(5): eval()'d code" giving a magic name for the function of "\0{closure}/tmp/y.php(5): eval()'d code0x0x7fa11ccb9ef7" where the addr is the absolute memory location in the string being evaluated. Hence in this scenario, each evaluation creates a new entry in the function_table, even though the closure is subsequently DTORed. In many ways this is the same behaviour are similar to create_function which generates magic names "\0Lambda%d" where the integer is the # of the lamda generated and again these build up in the function_table and are not GC'ed. The interesting Q is why isn't this always the behaviour? The reason is that the allocator includes an optimisation whereby if a string with an RC=1 is being replaced by a string of the same size then the memory is reused. If the new string contains a new closure function starting at the same offset then by accident the magic name will be the same as the previous (and different function). The compilation invokes the function zend_do_begin_function_declaration() and here the !is_method path does a zend_hash_update on the CG function_table. As the names happen to be the same, the update executes the function table DTOR on the previous entry cleaning it up. This accidental cleanup seems like a bug. I'll try to find an exploitable example. ------------------------------------------------------------------------ -- Edit this bug report at https://bugs.php.net/bug.php?id=64291&edit=1