Re: memory usage issues in PEAR 1.4.0

I probably wont go on much further on this, but It is a concern that I have heard from a few other developers, that the installer is growing very fat, however I havent got the time or inclination to dig into pear's code to understand the details or even fix it. I can understand how this has happened, a considerable amount of features have been added, but like all projects that evolve like this, the point where you have practically completed the main task is where the real challenges lie, optimizing, pruning etc (or throwing out the bathwater, as I've done before..). The installer looks like it has got to that stage now, but It would be conforting now, to hear a glowing ethusiasum to attack the problem of bloat as much as the problem of missing features. many of which sound essential, but may in hindsight be regarded as a non critical feature added to the core... These questions come to mind, by just by browsing the code, there looks like alot of places where non-essential features may have been escalated to must-haves. - validation ? is it necessary if the packager and installer versions match? how much validation is really essential at installation? - php code parsing? - is that code neccary for the installion part.? - xml handling? - is this parsing necessary at install time? (if packager / installer version match or are similar)? - channels? - is this code necessary for pure pear code?, how much of it is? could it be broken into a seperate package? - dependancies? - how many installs require complex dependancy resolution?, most, some or very few? ** these are semi-rehtorical questions - I dont expect an answer... (or particullaly want one..) at the end of the day, the most common situation (eg. installing a package from pear or pecl, where all the very simple dependancies are met), should probably struggle to require ~2Mb of memory..

We're only talking about the difference between 8MB and 11MB - is this really worth any sweat and tears to fix?

So when the next round of features get added, then bumping it to 16,32 etc. becomes ok... - 1.3.2 ran in < 4Mb, I dont think it's unrealistic to expect 1.4 to target <6Mb, (but like every mozilla release these days, the fact they are are lowering the resourse required is often more impressive than the increase in features..)

The only other idea the seems to come from looking at the code is that when packaging, storing a serialized verison of the xml file as well as the orginal, so XML parsing is only needed on old files..

This would actually increase memory consumption, unfortunately, as we would need to add code to check for the serialized versions.

eh? It would in a majority of cases, at present.. -> but in the future you would significantly reduce consuption by eliminating XML parsing except on packaging.., and eventually the majority of installs would require negligable code or parsing.

This could possibly remove alot of XML parsing, dependancy parsing validation etc. - How much validation is actually done on install BTW?

Quite a bit, to make sure that packages that are packaged with rogue older PEAR versions don't cause real problems, for instance. Every needed element is checked to make sure it is there,

Is all this validation needed? - Not knowing the installer inside out from memory, but it appears that PEAR/Validate.php is included at the install stage, it has a huge amount of code that is completely unnessary for install checking. Regards Alan

and unlike PEAR 1.3.5, every aspect of the file tags including the tasks and <replace> are validated on install-time. Greg P.S. I started playing around with a simplexml-based package.xml parser/validator, and it would just be about a million times faster and simpler. It would also be nice to save the registry as xml instead of using serialize(), so that hand-editing is possible, and corruption is easy to fix. Configuration stored as an .ini file for the same reasons would be great.

Some of these number illustrate the problems (although some are irrelivant.. ) 1.4.* > 1000 lines of code.

   LoC     words   Bytes
  1024    3366   40201 ./Command/Package.php
  1553    4586   56454 ./ChannelFile.php
  1086    3329   35733 ./Common.php
  1867    5732   62097 ./Config.php
  1079    3653   43692 ./Dependency2.php
  1200    3676   46248 ./Downloader.php
  1446    4357   58359 ./Downloader/Package.php
  1507    5407   59121 ./PackageFile/Generator/v2.php
  1089    3400   43516 ./PackageFile/Generator/v1.php
  1815    5130   73178 ./PackageFile/v2/Validator.php
  2916    8677  106936 ./PackageFile/v2.php
  1471    3892   48254 ./PackageFile/v1.php

1.3.* > 1000 lines of code.

  2040    6399   68959 ./Common.php
  1169    3593   34843 ./Config.php
  1039    3463   39984 ./Installer.php