Bug #10701 Updated: readfile usage on large files

ID: 10701 Updated by: sas Reported By: stephen@national-net.com Old-Status: Open Status: Closed Bug Type: Filesystem function related Operating system: PHP Version: 4.0.5 Assigned To: Comments: It is almost impossible to tell from the output of system commands like top why certain processes have such a huge memory footprint and what kind of memory they are associated with. In this case, you get the impression that the processes use much memory, because PHP maps files into the web-server's address space and sends them out in one quick move. These maps do not cause swapping though, as they can be removed from physical RAM without any swapping. Your problem is probably caused by using Apache for serving large static files. Every download ties up one Apache instance and hence the available RAM limits the number of users you can serve. Switching the file-server to e.g. thttpd would free up significant resources (RAM and CPU time). Previous Comments: --------------------------------------------------------------------------- [2001-05-18 11:10:47] sniper@php.net Reopened. This is not fixed. --Jani --------------------------------------------------------------------------- [2001-05-18 00:51:27] stephen@national-net.com You are kidding right? Nice way to take down a server. load average: 66.52, 33.25, 15.76 $fp = fopen("/web/sites/contentsite/".$f); while(!feof($fp)) { echo fgets($fd, 4096); } fclose($fp); --------------------------------------------------------------------------- [2001-05-18 00:28:19] rasmus@php.net What's to claim. This is a support question that belongs on one of the mailing lists and not in the bug database. But a hint. Don't use readfile(), fopen() the file and read it a bit at a time instead of sticking the entire thing in memory. --------------------------------------------------------------------------- [2001-05-17 17:29:33] stephen@national-net.com Anyone plan on claiming this? --------------------------------------------------------------------------- [2001-05-07 07:34:13] stephen@national-net.com Ok, this is a pretty intersting one, and I'm not even sure if it's a bug or just a memory limit thing. I am using a php wrapper for content files (.jpg|.gif|.asf|.mov|.ram), as you can guess, the movie files can be pretty large, upwards of 10MB, all getting read into memory, then spit back out. Couple that with a site that gets 10K+ visits a day and the server is on its knees with a load average of about 10 (when it gets to 25/30 things start swapping and it will dies not long after that.) here's the code used to wrap the content: <?php $mime_type = strtolower(strrchr($f,'.')); $mime_type_array = array( '.asf' => 'application/vnd.ms-asf', '.jpg' => 'image/jpeg', '.gif' => 'image/gif' ); if(!in_array($mime_type,array_keys($mime_type_array))) { header("Location: /error.php"); } $offset = 86400 * 3; header("Expires: ".gmdate("D, d M Y H:i:s GMT", time() + $offset)); header("Cache-Control: max-age=".$offset.", must-revalidate"); header("Last-modified : ".gmdate("D, d M Y H:i:s GMT", filemtime("/web/sites/contentsite/".$f))); header("Content-type: ".$mime_type_array[$mime_type]); header("Content-length: ".filesize("/web/sites/contentsite/".$f)); @readfile("/web/sites/contentsite/".$f); ?> so, I would pass an image or movie to the content file with a url like so: http://contentsite.com/content.php?f=movies/bigmovie.asf This is really just a heads up at this point, I know it will take you guys a little while to sort through this one, I'm not even sure it's a bug considering readfile is SUPPOSED to read a file into memory and spit it back out. I dunno, for now I'm going to do some .htaccess tricks where I force php to parse .htaccess files. If anyone has come across this or has any insight on wrapping content in php files, please email me at stephen@national-net.com Thanks! Stephen VanDyke PS - aside from that, great language, I love PHP :) --------------------------------------------------------------------------- The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online. ATTENTION! Do NOT reply to this email! To reply, use the web interface found at http://bugs.php.net/?id=10701&edit=2