Re: Arrays and performance

hi, In fact i had to handle the ODP dump on two occaisions the first time the results went into a mysql db, the second time it went into a series of files. On both occaisions i used SAX parsers. DOM would just roll over and die with this much of data. I placed code in the end element handler that would either save the data into a db or would save it to a file. In either case i only kept the data in memory for a short period. ie from the time the start element was detected through the character data handling until the end element was detected. (Obviously i am not talking of the root node here :-)) During the whole process you barely noticed the memory usage, however the disk usage still went up of course. Reading from disk 1 and writing to disk 2 does wonders! please let me know if you need any further clarifications. Pablo Gosse wrote:

Raditha Dissanayake wrote: [snip]The biggest XML job i have handled with PHP is parsing the ODP RDF dump which is around 700MB. Obviously arrays are out of the question in such a scenario, even though only one user will be accessing the script At a given moment. the ODP dump has a couple of million records[/snip] What was your solution for this, Raditha? How did you handle the parsing of such a large job? Cheers, Pablo

-- Raditha Dissanayake. ------------------------------------------------------------------------ http://www.radinks.com/sftp/ | http://www.raditha.com/megaupload Lean and mean Secure FTP applet with | Mega Upload - PHP file uploader Graphical User Inteface. Just 150 KB | with progress bar.