[Fwd: Re: phpdocumentor memory issue]
| From: | Greg Beaver | Date: | Mon, 03 Nov 2003 17:12:46 +0000 |
| Subject: | [Fwd: Re: phpdocumentor memory issue] | ||
| Groups: | php.pear.dev | ||
| Request: | Send a blank email to pear-dev+get-23207@lists.php.net to get a copy of this message | ||
didn't realize this was CCed to pear-dev
Greg
-------- Original Message --------
Subject: Re: phpdocumentor memory issue
Date: Mon, 03 Nov 2003 11:59:38 -0500
From: Greg Beaver <greg@chiaraquartet.net>
To: Klaus Guenther <klaus@capitalfocus.org>
References: <032c01c3a216$72fb7c10$5800a8c0@desk>
Hi Klaus,
The internal database that phpDocumentor uses in order to support some of the significant features in version 1.x takes up a tremendous amount of memory. In order to generate the full documentation for PEAR at phpdorks.net, Josh told me it peaked at 800 MB of memory usage. phpDocumentor itself requires about 120-220 depending on settings (sourcecode=on takes up lots of memory).
The tradeoff is between memory usage and speed. On slower processors, it can take days to generate docs if we use lots of disk access.
However, the rewrite which is currently taking place to create phpDocumentor 2.0 will fix these issues using some advanced caching for larger projects - the only time it will take any amount of time will be the first parse, and it will never take a large amount of memory, as we will be saving things in a database. Of course, this requires extra setup, so the default will still be to keep everything in memory, as this is much faster for small projects (the majority of our users).
Hope this helps in understanding what is going on. We have to keep track of every single item including line number, documentation, and so on. Plus an in-memory database of information for internal linking must be maintained (think LARGE array), and also the generation of indexes (both for each package, and all items parsed) temporarily take up a huge amount of memory (also a large array) before they are written to disk. Memory cannot be easily purged in case there are multiple output formats being generated at once.
All of these issues will be easily resolved in 2.0 :)
Greg
Klaus Guenther wrote:
Hi... Just was trying to generate the docs for my pear install... Everything was going fine until I got this error Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 40 bytes) in /usr/share/php/PhpDocumentor/phpDocumentor/phpDocumentorTWordParser.inc on line 269 Normally I wouldn't let PHP use so much memory. And I don't think it's appropriate for it to use so much. Here's the commandline: # phpdoc -t /home/www/web1/html/page/docs -d /usr/share/php/ SuSE 8.1; php 4.3.4RC2 Here's the output of pear list Installed packages: ===================Package Version State Archive_Tar 1.1 stable Benchmark 1.2.1 stable Cache_Lite 1.2 stable Config 1.9 stable Console_Getopt 1.0 stable Crypt_CHAP 0.8.5 alpha Crypt_RC4 1.0.2 stable DB 1.5.0RC2 stable Date 1.3.1beta beta File 1.0.3 stable File_Passwd 0.9.5 beta HTML_CSS 0.3.0 beta HTML_Common 1.2.1 stable HTML_Page 2.0.0RC2 beta HTML_Progress 0.6.2 stable HTML_QuickForm 3.1.1 stable HTML_Select 1.2 beta HTML_Table 1.5 stable HTTP 1.2.1 stable HTTP_Session 0.4 beta LiveUser 0.9 beta Log 1.7.1 stable MDB 1.1.4RC3 devel Mail 1.1.2 stable Mail_Mime 1.2.1 stable Math_Stats 0.9.0beta3 beta Net_POP3 1.2 stable Net_SMTP 1.2.3 stable Net_Socket 1.0.1 stable Numbers_Roman 0.1 stable PEAR 1.3b3 beta PEAR_PackageFileMana 1.0 stablegerPHPUnit 0.6.2 stable PhpDocumentor 1.2.3 stable Translation 1.2.5 stable XML_DTD 0.4.1 alpha XML_Parser 1.0.1 stable XML_RPC 1.0.4 stable XML_Statistics 0.1 beta XML_Tree 2.0b2 beta XML_Util 0.5.1 stableIt's not an outrageous number of packages to have installed. Any ideas what may be going wrong? Thanks, Klaus