Bug #80158 [Com]: Memory leak when xml nodes are stored into local array

Edit report at https://bugs.php.net/bug.php?id=80158&edit=1 ID: 80158 Comment by: efbiaiinzinz at hotmail dot com Reported by: efbiaiinzinz at hotmail dot com Summary: Memory leak when xml nodes are stored into local array Status: Feedback Type: Bug Package: SimpleXML related Operating System: Linux PHP Version: 7.4.10 Block user comment: N Private report: N New Comment: I'm having bit of a trouble trying to debug it more due to the "harmless" leak because it makes it hard to pinpoint exact place and behaviour where leak is occurring. I confirmed overnight that this leak issue also exists in 7.3.22. I modified the sample script and changed: $mem1 = memory_get_usage(); testFile($file); $mem2 = memory_get_usage(); echo "$mem1\n$mem2\n"; to: $mem1 = memory_get_usage(); testFile($file); $mem2 = memory_get_usage(); $str500KB = str_repeat('1234567890', 50 * 1024); $mem3 = memory_get_usage(); $str10MB = str_repeat('1234567890', 1024 * 1024); $mem4 = memory_get_usage(); testFile($file); $mem5 = memory_get_usage(); echo "$mem1\n$mem2\n$mem3\n$mem4\n$mem5\n"; I see output (running through apache) 391936 908032 1424128 11914008 11914008 Creating new strings does not reuse the memory but rather increases the memory usage on top of existing number, but rerunning the same simplexml elemnt parser + loop does not increase the total memory indeed. Do you have any pointers on how to debug it more or if there is some known edge case that can cause the object store harmless leak turn into an actual leak? Right now I have one codepath with xml input of 50MB and according to memory_get_usage() running it twice in row results ~72MB getting lost for each run and another codepath that uses 80MB json file as input and first run shows memory difference 0.11MB (loads multiple classes) and second run shows memory difference 0.00MB as expected. I will try to narrow it down some more this weekend. Previous Comments: ------------------------------------------------------------------------ [2020-10-02 12:05:47] efbiaiinzinz at hotmail dot com I'll try to make a more complex sample, but something strange is happening still . The full system where it started occurring does lots of reading from different datapoints (some xml, some json, some csv). There is a common Base class extended by different per-endpoint implementations, that map the specific structure into common internal format and then pass it on to additional handlers that store changes into db. I have added logging and the handlers that use bigger CSV files (50MB plus) and read 100k rows and do lots of interactions show memory differece 0.01-0.05MB (class definition loading takes memory). When measuring memory after XML parsers, they all report memory "leag" counted in megabytes, worst case 80MB+ And out of memory has happened due to it already, so it seems to be eprmanent leak affecting new allocations. I'll try to run the detailed measurements when running cron with 7.3 version and then can report the difference in more detail. ------------------------------------------------------------------------ [2020-10-01 14:50:01] nikic@php.net > When looping over xml nodes and storing the node objects temporarily into a local array, some > memory is never released and doing it many times in a row with different files will result in out of > memory error. From a cursory look, this looks like an object store "leak", which means that memory usage is proportional to the maximum number of objects live at the same time. This is generally harmless as it does not result in a persistent leak and the memory is reused when creating new objects. However, it also means that "doing it many times in a row with different files will result in out of memory error" does not hold, assuming you don't keep objects alive between files. ------------------------------------------------------------------------ [2020-09-27 21:34:47] efbiaiinzinz at hotmail dot com Description: ------------ When looping over xml nodes and storing the node objects temporarily into a local array, some memory is never released and doing it many times in a row with different files will result in out of memory error. Provided sample shows that ~516096 bytes is lost. When I change the test script line $items[(int)(string)$item->id] = $item; to $items[(int)(string)$item->id] = (string)$item->name; or even to $items[(int)(string)$item->id] = $item->asXML(); Then memory amount before and after testFile() call is exactly the same. Test script: --------------- function generate($counter) { $file = tempnam(sys_get_temp_dir(), 'xml'); $f = fopen($file, 'wb'); fwrite($f, '<?xml version="1.0" encoding="UTF-8"?><items>' . "\n"); for ($i = 1; $i <= $counter; $i++) fwrite($f, '<item id="' . $i . '"><id>' . $i . '</id><name>item ' . $i . '</name></item>' . "\n"); fwrite($f, '</items>'); fclose($f); return $file; } function testFile($file) { $xml = simplexml_load_file($file); $items = []; foreach ($xml->item as $item) $items[(int)(string)$item->id] = $item; } $file = generate(50000); $mem1 = memory_get_usage(); testFile($file); $mem2 = memory_get_usage(); echo "$mem1\n$mem2\n"; unlink($file); Expected result: ---------------- The test should print two exactly same numbers, separated by linebreak. Actual result: -------------- Second number is ~516 000 larger which means at least 0.5MB of memory leaked somehow. ------------------------------------------------------------------------ -- Edit this bug report at https://bugs.php.net/bug.php?id=80158&edit=1