Bug #74072 [Opn->Dup]: Memory leaks with setter causing out of memory
Edit report at https://bugs.php.net/bug.php?id=74072&edit=1
ID: 74072
Updated by: nikic@php.net
Reported by: ealexs at gmail dot com
Summary: Memory leaks with setter causing out of memory
-Status: Open
+Status: Duplicate
Type: Bug
Package: Reproducible crash
Operating System: Debian 8
PHP Version: 7.0.15
Block user comment: N
Private report: N
New Comment:
Duplicate of bug #65340.
Previous Comments:
------------------------------------------------------------------------
[2017-02-16 11:22:55] torben at dannhauer dot info
I can reproduce this bug with the below test script.
I have a error in a production server which seems to be the same but I'm not sure.
On my side, your testscript also fails with xdebug modul loaded, however my production error does
disappear with Xdebug module loaded.
------------------------------------------------------------------------
[2017-02-16 07:53:24] ealexs at gmail dot com
updated the "Package"
------------------------------------------------------------------------
[2017-02-11 12:03:29] ealexs at gmail dot com
I have managed to create a test script that reproduces the bug
by calling "test_set_by_method($max)" the execution works fine
Using test_set_by_magic_setter($max) it will run out of memory
<?php
header("Content-type: text");
ini_set('memory_limit', '32M');
// warm up
$obj = new TestGS();
$max = 200000;
// $t1 = microtime(true);
// this will finish ok
test_set_by_method($max);
// $set_time = microtime(true) - $t1;
// $t1 = microtime(true);
// this will cause Allowed memory size ... exhausted
test_set_by_magic_setter($max);
// $set_time = microtime(true) - $t1;
// end warm up
function test_set_by_method($max = 200000)
{
$t1 = microtime(true);
$val = 1;
$bag = [];
echo "START SET BY METHOD =======================================\n\n";
for ($i = 0; $i < $max; $i++)
{
$obj = new TestGS();
$obj->setProperty_001($val++);
$obj->setProperty_002($val++);
$obj->setProperty_003($val++);
$obj->setProperty_004($val++);
$obj->setProperty_005($val++);
$obj->setProperty_006($val++);
$obj->setProperty_007($val++);
$obj->setProperty_008($val++);
$obj->setProperty_009($val++);
$obj->setProperty_010($val++);
$obj->setProperty_011($val++);
$obj->setProperty_012($val++);
$obj->setProperty_013($val++);
$obj->setProperty_014($val++);
$obj->setProperty_015($val++);
$obj->setProperty_016($val++);
$obj->setProperty_017($val++);
$obj->setProperty_018($val++);
$obj->setProperty_019($val++);
$obj->setProperty_020($val++);
// every 10 k we dump
if ((int)(floor($i/10000)*10000) === (int)$i)
{
echo "Memory usage [{$i}]: ".(memory_get_usage(true)/1024)." KB\n";
}
$bag[] = $obj;
}
echo "Memory usage [{$i}]: ".(memory_get_usage(true)/1024)." KB\n";
echo "DONE SET BY METHOD =======================================\n\n";
return [$bag, $val];
}
function test_set_by_magic_setter($max = 200000)
{
$val = 1;
$bag = [];
echo "START SET BY MAGIC SETTER =======================================\n\n";
for ($i = 0; $i < $max; $i++)
{
$obj = new TestGS();
$obj->property_001 = $val++;
$obj->property_002 = $val++;
$obj->property_003 = $val++;
$obj->property_004 = $val++;
$obj->property_005 = $val++;
$obj->property_006 = $val++;
$obj->property_007 = $val++;
$obj->property_008 = $val++;
$obj->property_009 = $val++;
$obj->property_010 = $val++;
$obj->property_011 = $val++;
$obj->property_012 = $val++;
$obj->property_013 = $val++;
$obj->property_014 = $val++;
$obj->property_015 = $val++;
$obj->property_016 = $val++;
$obj->property_017 = $val++;
$obj->property_018 = $val++;
$obj->property_019 = $val++;
$obj->property_020 = $val++;
// every 10 k we dump
if ((int)(floor($i/10000)*10000) === (int)$i)
{
echo "Memory usage [{$i}]: ".(memory_get_usage(true)/1024)." KB\n";
}
$bag[] = $obj;
}
echo "Memory usage [{$i}]: ".(memory_get_usage(true)/1024)." KB\n";
echo "END SET BY MAGIC SETTER =======================================\n\n";
return [$bag, $val];
}
function test_get_by_method($bag)
{
// GET TEST
foreach ($bag as $obj)
{
if ($obj->getProperty_001()) $val++;
if ($obj->getProperty_002()) $val++;
if ($obj->getProperty_003()) $val++;
if ($obj->getProperty_004()) $val++;
if ($obj->getProperty_005()) $val++;
if ($obj->getProperty_006()) $val++;
if ($obj->getProperty_007()) $val++;
if ($obj->getProperty_008()) $val++;
if ($obj->getProperty_009()) $val++;
if ($obj->getProperty_010()) $val++;
if ($obj->getProperty_011()) $val++;
if ($obj->getProperty_012()) $val++;
if ($obj->getProperty_013()) $val++;
if ($obj->getProperty_014()) $val++;
if ($obj->getProperty_015()) $val++;
if ($obj->getProperty_016()) $val++;
if ($obj->getProperty_017()) $val++;
if ($obj->getProperty_018()) $val++;
if ($obj->getProperty_019()) $val++;
if ($obj->getProperty_020()) $val++;
/*
if ($obj->property_001) $val++;
if ($obj->property_002) $val++;
if ($obj->property_003) $val++;
if ($obj->property_004) $val++;
if ($obj->property_005) $val++;
if ($obj->property_006) $val++;
if ($obj->property_007) $val++;
if ($obj->property_008) $val++;
if ($obj->property_009) $val++;
if ($obj->property_010) $val++;
if ($obj->property_011) $val++;
if ($obj->property_012) $val++;
if ($obj->property_013) $val++;
if ($obj->property_014) $val++;
if ($obj->property_015) $val++;
if ($obj->property_016) $val++;
if ($obj->property_017) $val++;
if ($obj->property_018) $val++;
if ($obj->property_019) $val++;
if ($obj->property_020) $val++;*/
}
}
// 967 ms | 390 ms | 2.4 more time
// for compile / provisioning it's fine ... a lot of things will be internal
/*echo "\n\nGET time for {$max} objs : ", round($get_time*1000, 4), " ms";
echo "\n\nSET time for {$max} objs : ", round($set_time*1000, 4), " ms";
echo "\n\nTotal time: ", round((microtime(true) - $t1)*1000, 4), " ms";
*/
class TestGS
{
protected $property_001;
protected $property_002;
protected $property_003;
protected $property_004;
protected $property_005;
protected $property_006;
protected $property_007;
protected $property_008;
protected $property_009;
protected $property_010;
protected $property_011;
protected $property_012;
protected $property_013;
protected $property_014;
protected $property_015;
protected $property_016;
protected $property_017;
protected $property_018;
protected $property_019;
protected $property_020;
public function getProperty_001() { return $this->property_001; }
public function getProperty_002() { return $this->property_002; }
public function getProperty_003() { return $this->property_003; }
public function getProperty_004() { return $this->property_004; }
public function getProperty_005() { return $this->property_005; }
public function getProperty_006() { return $this->property_006; }
public function getProperty_007() { return $this->property_007; }
public function getProperty_008() { return $this->property_008; }
public function getProperty_009() { return $this->property_009; }
public function getProperty_010() { return $this->property_010; }
public function getProperty_011() { return $this->property_011; }
public function getProperty_012() { return $this->property_012; }
public function getProperty_013() { return $this->property_013; }
public function getProperty_014() { return $this->property_014; }
public function getProperty_015() { return $this->property_015; }
public function getProperty_016() { return $this->property_016; }
public function getProperty_017() { return $this->property_017; }
public function getProperty_018() { return $this->property_018; }
public function getProperty_019() { return $this->property_019; }
public function getProperty_020() { return $this->property_020; }
public function setProperty_001(int $value) { $this->property_001 = $value; }
public function setProperty_002(int $value) { $this->property_002 = $value; }
public function setProperty_003(int $value) { $this->property_003 = $value; }
public function setProperty_004(int $value) { $this->property_004 = $value; }
public function setProperty_005(int $value) { $this->property_005 = $value; }
public function setProperty_006(int $value) { $this->property_006 = $value; }
public function setProperty_007(int $value) { $this->property_007 = $value; }
public function setProperty_008(int $value) { $this->property_008 = $value; }
public function setProperty_009(int $value) { $this->property_009 = $value; }
public function setProperty_010(int $value) { $this->property_010 = $value; }
public function setProperty_011(int $value) { $this->property_011 = $value; }
public function setProperty_012(int $value) { $this->property_012 = $value; }
public function setProperty_013(int $value) { $this->property_013 = $value; }
public function setProperty_014(int $value) { $this->property_014 = $value; }
public function setProperty_015(int $value) { $this->property_015 = $value; }
public function setProperty_016(int $value) { $this->property_016 = $value; }
public function setProperty_017(int $value) { $this->property_017 = $value; }
public function setProperty_018(int $value) { $this->property_018 = $value; }
public function setProperty_019(int $value) { $this->property_019 = $value; }
public function setProperty_020(int $value) { $this->property_020 = $value; }
public function __set($name, $value)
{
// both of these fail
// this will cause Allowed memory size ... exhausted
// $this->{'set'.$name}($value);
// this will also cause Allowed memory size ... exhausted
$this->$name = $value;
}
}
------------------------------------------------------------------------
[2017-02-10 10:39:00] ealexs at gmail dot com
Description:
------------
We have a framework with an ORM within. The MODEL object has a setter in it that causes the
problems.
We pull ~100.000 records from a DB (mysqli) in batches of 1000.
I use "memory_get_usage(true)" to monitor the memory usage
after each batch.
The behaiviour is random.
After the first batches memory_get_usage(true) = 2 MB.
Then it starts to grow until it runs out of memory (set at 32MB atm).
In rare cases (refreshing) it finishes.
It always does the same thing! (no data changed - devel DB).
If the setter (public function __set) is removed, or we avoid beeing called,
then it always runs and memory_get_usage(true) stays at 2 MB. Always.
<?php
public function __set($property, $value)
{
return $this->set($property, $value);
}
public function set($property, $value = null, $transform_with = null, $requester = null, $reason =
null)
{
if ($property{0} === "_")
{
$this->$property = $value;
return;
}
// manage saving the old property
if ((!(isset($this->_ols[$property]) || ($this->_ols && array_key_exists($property,
$this->_ols)))) && (($value === null) || ($this->$property !== $value)))
$this->_ols[$property] = $this->$property;
// end old
$set = $this->getModelType()->properties[$property]->setter;
if ($set)
$this->$set($value);
else
$this->$property = $value;
}
?>
I was unable to isolate the bug outside our framework atm.
Here is Valgrind's output (if the setter is removed or not called these elements vanishes)
==6720==
==6720== 2,299,968 bytes in 7,986 blocks are possibly lost in loss record 9,815 of 9,817
==6720== at 0x4C28C20: malloc (vg_replace_malloc.c:296)
==6720== by 0x831CD8: __zend_malloc (zend_alloc.c:2864)
==6720== by 0x863DF6: zend_hash_real_init_ex (zend_hash.c:140)
==6720== by 0x863DF6: zend_hash_check_init (zend_hash.c:163)
==6720== by 0x863DF6: _zend_hash_add_or_update_i (zend_hash.c:563)
==6720== by 0x863DF6: _zend_hash_add (zend_hash.c:638)
==6720== by 0x88D53C: zend_hash_add_mem (zend_hash.h:570)
==6720== by 0x88D53C: zend_get_property_guard (zend_object_handlers.c:503)
==6720== by 0x88FB76: zend_std_write_property (zend_object_handlers.c:667)
==6720== by 0x8B71E3: zend_assign_to_object (zend_execute.c:1234)
==6720== by 0x8B71E3: ZEND_ASSIGN_OBJ_SPEC_CV_CONST_HANDLER (zend_vm_execute.h:32028)
==6720== by 0x89584A: execute_ex (zend_vm_execute.h:414)
==6720== by 0x8E9566: zend_execute (zend_vm_execute.h:458)
==6720== by 0x857EF3: zend_execute_scripts (zend.c:1437)
==6720== by 0x7FB23F: php_execute_script (main.c:2492)
==6720== by 0x8EB1F9: do_cli (php_cli.c:977)
==6720== by 0x449796: main (php_cli.c:1347)
==6720==
==6720==
==6720== 18,452,736 bytes in 8,009 blocks are possibly lost in loss record 9,817 of 9,817
==6720== at 0x4C28C20: malloc (vg_replace_malloc.c:296)
==6720== by 0x831CD8: __zend_malloc (zend_alloc.c:2864)
==6720== by 0x862C3F: zend_hash_real_init_ex (zend_hash.c:140)
==6720== by 0x862C3F: zend_hash_real_init (zend_hash.c:205)
==6720== by 0x88E192: rebuild_object_properties (zend_object_handlers.c:82)
==6720== by 0x8900B2: zend_std_write_property (zend_object_handlers.c:709)
==6720== by 0x8BCAF2: zend_assign_to_object (zend_execute.c:1234)
==6720== by 0x8BCAF2: ZEND_ASSIGN_OBJ_SPEC_UNUSED_CV_HANDLER (zend_vm_execute.h:26390)
==6720== by 0x89584A: execute_ex (zend_vm_execute.h:414)
==6720== by 0x849650: zend_call_function (zend_execute_API.c:858)
==6720== by 0x874A32: zend_call_method (zend_interfaces.c:104)
==6720== by 0x88FBD9: zend_std_call_setter (zend_object_handlers.c:216)
==6720== by 0x88FBD9: zend_std_write_property (zend_object_handlers.c:674)
==6720== by 0x8B71E3: zend_assign_to_object (zend_execute.c:1234)
==6720== by 0x8B71E3: ZEND_ASSIGN_OBJ_SPEC_CV_CONST_HANDLER (zend_vm_execute.h:32028)
==6720== by 0x89584A: execute_ex (zend_vm_execute.h:414)
==6720== by 0x8E9566: zend_execute (zend_vm_execute.h:458)
==6720== by 0x857EF3: zend_execute_scripts (zend.c:1437)
==6720== by 0x7FB23F: php_execute_script (main.c:2492)
==6720== by 0x8EB1F9: do_cli (php_cli.c:977)
==6720== by 0x449796: main (php_cli.c:1347)
==6720==
------------------------------------------------------------------------
--
Edit this bug report at https://bugs.php.net/bug.php?id=74072&edit=1
Thread (8 messages)