Bug #80158 [Com]: Memory leak when xml nodes are stored into local array
Edit report at https://bugs.php.net/bug.php?id=80158&edit=1
ID: 80158
Comment by: efbiaiinzinz at hotmail dot com
Reported by: efbiaiinzinz at hotmail dot com
Summary: Memory leak when xml nodes are stored into local
array
Status: Feedback
Type: Bug
Package: SimpleXML related
Operating System: Linux
PHP Version: 7.4.10
Block user comment: N
Private report: N
New Comment:
I'll try to make a more complex sample, but something strange is happening still
.
The full system where it started occurring does lots of reading from different datapoints (some xml,
some json, some csv).
There is a common Base class extended by different per-endpoint implementations, that map the
specific structure into common internal format and then pass it on to additional handlers that store
changes into db.
I have added logging and the handlers that use bigger CSV files (50MB plus) and read 100k rows and
do lots of interactions show memory differece 0.01-0.05MB (class definition loading takes memory).
When measuring memory after XML parsers, they all report memory "leag" counted in
megabytes, worst case 80MB+
And out of memory has happened due to it already, so it seems to be eprmanent leak affecting new
allocations.
I'll try to run the detailed measurements when running cron with 7.3 version and then can
report the difference in more detail.
Previous Comments:
------------------------------------------------------------------------
[2020-10-01 14:50:01] nikic@php.net
> When looping over xml nodes and storing the node objects temporarily into a local array, some
> memory is never released and doing it many times in a row with different files will result in out of
> memory error.
From a cursory look, this looks like an object store "leak", which means that memory usage
is proportional to the maximum number of objects live at the same time. This is generally harmless
as it does not result in a persistent leak and the memory is reused when creating new objects.
However, it also means that "doing it many times in a row with different files will result in
out of memory error" does not hold, assuming you don't keep objects alive between files.
------------------------------------------------------------------------
[2020-09-27 21:34:47] efbiaiinzinz at hotmail dot com
Description:
------------
When looping over xml nodes and storing the node objects temporarily into a local array, some memory
is never released and doing it many times in a row with different files will result in out of memory
error.
Provided sample shows that ~516096 bytes is lost.
When I change the test script line
$items[(int)(string)$item->id] = $item;
to
$items[(int)(string)$item->id] = (string)$item->name;
or even to
$items[(int)(string)$item->id] = $item->asXML();
Then memory amount before and after testFile() call is exactly the same.
Test script:
---------------
function generate($counter) {
$file = tempnam(sys_get_temp_dir(), 'xml');
$f = fopen($file, 'wb');
fwrite($f, '<?xml version="1.0"
encoding="UTF-8"?><items>' . "\n");
for ($i = 1; $i <= $counter; $i++)
fwrite($f, '<item id="' . $i . '"><id>' . $i .
'</id><name>item ' . $i . '</name></item>' .
"\n");
fwrite($f, '</items>');
fclose($f);
return $file;
}
function testFile($file) {
$xml = simplexml_load_file($file);
$items = [];
foreach ($xml->item as $item)
$items[(int)(string)$item->id] = $item;
}
$file = generate(50000);
$mem1 = memory_get_usage();
testFile($file);
$mem2 = memory_get_usage();
echo "$mem1\n$mem2\n";
unlink($file);
Expected result:
----------------
The test should print two exactly same numbers, separated by linebreak.
Actual result:
--------------
Second number is ~516 000 larger which means at least 0.5MB of memory leaked somehow.
------------------------------------------------------------------------
--
Edit this bug report at https://bugs.php.net/bug.php?id=80158&edit=1
Thread (6 messages)