Re: XML in PHP5

I like the proposals for creating a replacement for DOM XML. There is still a need for a super fast SAXy XML interface and I think it makes sense to have two interfaces. Let me explain with an example. When I display one document, I want all the data first then I want to pick bits out of the whole document to build a heading then I want to display the whole lot on the page. It makes sense to read the whole document in one hit, have a structure I can access at random, then have a method to convert the structure to a display format. When I have 385 million documents on file, I also want a different approach. In one case, I set up a system for managing documents with a identifying header and a summary footer. Using normal XML stuff, I put the summary at the start of the XML DTD. That meant in most cases the summary is in the first 100 bytes of documents averaging over 8 Kb. No matter what operating system or file system, the summary happens to be in the first physical IO read. If I use SAX through the current XML interface, I can read just the first 100 bytes or the first 512 byte disk sector of each document and skim through the documents quickly. That lets me search the document summaries without having to extract the summaries to traditional database tables. I ran tests using SAX through XML and compared the result to a MySQL based approach. The SAX XML skim headers was a few times slower than a search of MySQL but around 100 times faster than using the DOM "read all the document before processing" approach. Using Oracle with all of Oracle's overheads and no special tuning produced a speed similar to the SAX skim approach. Overall the ability to skim the first few bytes of an XML document meant I could, on my tired old PC, search 100,000 documents in a few seconds instead on 20 minutes. The difference would be enormous if the documents were multipage documents occupying many file system allocation units which, in ext2, is anything over 8Kb. The other instance that requires special attention is the case with millions of records in one XML file. If you want one record out of millions and that record is <beer><name>Coopers</name><description>Brilliant</description></beer>, you do not want to read through to the end of file to verify the file is valid XML. All you want is to validate the format up to the end of the </beer> tag for the Coopers record. That means a way of telling the read process to stop reading once the record is found. This case also needs a way of throwing away the unneeded records. Every time you hit </beer> and decide the beer record is not the one you want, you want to throw away the existing data from the level of beer down. You want to do that without throwing away other parts of the structure. As far as I can tell, the current XSLT and Xpath approaches do not have a way of saying unset('/drinks/beer/name[!='Coopers']') I like the idea of a new DOMish interface for whole of document processing but still want a SAXy approach for speed reading collections of XML files.