Re: readfile() memory usage

Thanks all. So it sounds like the answer is: 1) readfile() has always been memory-safe as far as PHP is concerned. 2) Because it uses mmap(), GFL trying to understand its memory usage from top. 3) Operating systems have gotten better at such things in the past decade. 4) So given #2 and #3, the "readfile() will kill your memory, don't use it" line is a persistent urban legend that belongs on Snopes as debunked. Looping on fread() for performance is a red herring. Is that an accurate summary? If so, I will blog my benchmark results and this conversation. --Larry Garfield On 4/30/12 5:33 AM, Sherif Ramadan wrote:

Mmapping of course uses memory, but the memory used here is not from PHP's memory manager, it's memory that's already used for the O/S cache. The memory mapping used here does not enforce loading the file contents into O/S cache; it just gets a virtual address into the O/S cache. If the actual file contents are not yet in O/S cache, the O/S will hit a page fault and load the pages into memory. Apache Server uses the same mechanism to serve files. Maybe the "user confusion" about memory usage comes from that fact (they see lots of *virtual* memory used by PHP when viewed in top). I know this user confusion from my work in the Apache Lucene/Solr project, where one option (used on 64 bit operating systems) is to memory-map the whole Lucene full-text index. Users then see hundreds of Gigabytes of "virtual memory" usage in TOP / Windows Task Manager and are afraid of running their machine out of memory. This is always hard to explain to people that are not used to the term "virtual memory".

That's a very good point and a detailed look at the stack can show some of the underlying mechanics coming from readfile and how its pretty much is just implemented like fpassthru delegating a stream, as you said. http://i.imgur.com/PWiTv.png

About php memory usage, one has to use an external tools to actually see

this

memory usage as it is not managed by the zend memory manager.

Of course...

Also, running valgrind and taking a closer look at what memory blocks PHP is allocating here it can be better determined what's leaking and what isn't, of course... http://pastebin.com/LSQcUsBL I can certainly attest to users being deceived by memory readings in top in the past. It can be very deceiving if you think look at what free memory top or free shows you especially after some huge allocation. There's a clear difference here though between the Zend memory manager allocating these blocks in PHP and what's going on with readfile. The memory the system knows is available to it isn't being tied up in this case.