#45291 [NEW]: Memory failure in very large XML file

From: gilbert dot musnik at fr dot adp dot com Operating system: Windows PHP version: 5.2.6 PHP Bug Type: SimpleXML related Bug description: Memory failure in very large XML file Description: ------------ Hello, It's been days I am trying to sort out a problem which pops up when I process a huge XML file. In "reproduce code" windows is a trivial script which allocates memory by making an array very big. Assuming a memory_limit of 1024M in <php.ini>, if I don't load the huge XML file, the script crashes as expected with a "memory exhausted" fatal error diagnostic (see expected results window). However, if I do load my huge XML file with simplexml_load_file(around 300 MB!), the script crashes with a "Out of memory" fatal error diagnostic. By the way, it doesn't matter if memory_limit is set to 512M or 2048M, the script stumbles on the same "allocated" figure of 332 MB (see actual results windows). The strangest thing is that before I start swelling the array, the memory usage is almost the same whether ot not simplexml_load_file("VERY_BIG_FILE.XML") is invoked or not. It looks as if memory allocated by simplexml_load_file() routine is NOT taken into account in overall PHP memory usage and memory limit management. Thanks in advance for your feedback on that, Gilbert Musnik Reproduce code: --------------- <?php $flag_xml = true; $now = Date("Y/m/d - H:i:s"); if ($flag_xml) { print ("[ $now ] Before simplexml_load_file\n"); $f_inp = "VERY_BIG_FILE.xml"; // mine is 300 MB $ll0 = time(); $xml = simplexml_load_file ($f_inp); $ll1 = time() - $ll0; $now = Date("Y/m/D - H:i:s"); print ("[ $now ] After simplexml_load_file: $ll1 secondes\n"); } else { print ("[ $now ] NO simplexml_load_file\n"); } $bufrs = array(); $nmax = 150000000; for ($ii = 0; $ii < $nmax; $ii ++) { $bufr = str_repeat ("ABCDEFGHIJ", 1); array_push ($bufrs, $bufr); $jj = $ii % 1000000; if ($jj == 0) { $xx = number_format ($ii); $qq = number_format (memory_get_usage()); $memory_limit = ini_get('memory_limit'); print ("$xx - memory usage=$qq - memory_limit=$memory_limit)\n"); } }; ?> Expected result: ---------------- N:\>\php-5.2.6-Win32\php.exe -c \MD_config\php.ini \MD_config\test_mem.php [ 2008/06/16 - 18:06:10 ] No simplexml_load_file 0 - memory usage=64,552 - memory_limit=1024M) 1,000,000 - memory usage=100,259,664 - memory_limit=1024M) 2,000,000 - memory usage=200,454,016 - memory_limit=1024M) 3,000,000 - memory usage=304,842,624 - memory_limit=1024M) 4,000,000 - memory usage=400,842,624 - memory_limit=1024M) 5,000,000 - memory usage=513,619,840 - memory_limit=1024M) 6,000,000 - memory usage=609,619,840 - memory_limit=1024M) 7,000,000 - memory usage=705,619,840 - memory_limit=1024M) 8,000,000 - memory usage=801,619,840 - memory_limit=1024M) 9,000,000 - memory usage=931,174,272 - memory_limit=1024M) 10,000,000 - memory usage=1,027,174,272 - memory_limit=1024M) Fatal error: Allowed memory size of 1073741824 bytes exhausted (tried to allocate 35 bytes) in N:\md_config\test_mem.php on line 23 Actual result: -------------- N:\>\php-5.2.6-Win32\php.exe -c \MD_config\php.ini \MD_config\test_mem.php [ 2008/06/16 - 18:05:06 ] Before simplexml_load_file [ 2008/06/Mon - 18:05:33 ] After simplexml_load_file: 27 secondes 0 - memory usage=65,472 - memory_limit=1024M) 1,000,000 - memory usage=100,260,376 - memory_limit=1024M) 2,000,000 - memory usage=200,454,728 - memory_limit=1024M) 3,000,000 - memory usage=304,843,336 - memory_limit=1024M) Fatal error: Out of memory (allocated 332660736) (tried to allocate 35 bytes) in N:\md_config\test_mem.php on line 23 -- Edit bug report at http://bugs.php.net/?id=45291&edit=1 -- Try a CVS snapshot (PHP 5.2): http://bugs.php.net/fix.php?id=45291&r=trysnapshot52 Try a CVS snapshot (PHP 5.3): http://bugs.php.net/fix.php?id=45291&r=trysnapshot53 Try a CVS snapshot (PHP 6.0): http://bugs.php.net/fix.php?id=45291&r=trysnapshot60 Fixed in CVS: http://bugs.php.net/fix.php?id=45291&r=fixedcvs Fixed in release: http://bugs.php.net/fix.php?id=45291&r=alreadyfixed Need backtrace: http://bugs.php.net/fix.php?id=45291&r=needtrace Need Reproduce Script: http://bugs.php.net/fix.php?id=45291&r=needscript Try newer version: http://bugs.php.net/fix.php?id=45291&r=oldversion Not developer issue: http://bugs.php.net/fix.php?id=45291&r=support Expected behavior: http://bugs.php.net/fix.php?id=45291&r=notwrong Not enough info: http://bugs.php.net/fix.php?id=45291&r=notenoughinfo Submitted twice: http://bugs.php.net/fix.php?id=45291&r=submittedtwice register_globals: http://bugs.php.net/fix.php?id=45291&r=globals PHP 4 support discontinued: http://bugs.php.net/fix.php?id=45291&r=php4 Daylight Savings: http://bugs.php.net/fix.php?id=45291&r=dst IIS Stability: http://bugs.php.net/fix.php?id=45291&r=isapi Install GNU Sed: http://bugs.php.net/fix.php?id=45291&r=gnused Floating point limitations: http://bugs.php.net/fix.php?id=45291&r=float No Zend Extensions: http://bugs.php.net/fix.php?id=45291&r=nozend MySQL Configuration Error: http://bugs.php.net/fix.php?id=45291&r=mysqlcfg