#29922 [Asn->Csd]: Huge memory usage in file copy of big files

ID: 29922 Updated by: iliaa@php.net Reported By: info at pion dot xs4all dot nl -Status: Assigned +Status: Closed Bug Type: *Directory/Filesystem functions Operating System: Linux PHP Version: 5.0.1 Assigned To: wez New Comment: This bug has been fixed in CVS. Snapshots of the sources are packaged every three hours; this change will be in the next snapshot. You can grab the snapshot at http://snaps.php.net/. Thank you for the report, and for helping us make PHP better. Previous Comments: ------------------------------------------------------------------------ [2004-09-10 23:53:30] ben at prince dot org Just so we can be clear, while I understand this affects readfile() as well as copy(), does it also affect include()? ------------------------------------------------------------------------ [2004-09-02 14:53:18] wez@php.net IIRC, I added a limit for this kind of thing for readfile() (aka php_stream_passthru), which could be applied here. Discussions on the list (with Andi) suggested mmap'ing chunks of 1MB and doing it that way. If someone wants to cook up a patch for either of these, that's fine by me; I'm not sure how soon I'll be able to look into it. ------------------------------------------------------------------------ [2004-09-02 08:19:24] derick@php.net I saw a similar (but better) patch by Wez. Wez, can you have a look? ------------------------------------------------------------------------ [2004-09-01 22:19:57] info at pion dot xs4all dot nl I have a small patch for the 5.1.0-dev which solves the problem but: This patch says a normal copy should be used for files bigger then 32 MB. But this is an rather arbitrary value choosen by me. Here follows the patch: -diff -Naur org/php5-200409011830/main/streams/streams.c php5-200409011830/main/streams/streams.c --- org/php5-200409011830/main/streams/streams.c 2004-08-31 14:30:59.000000000 +0200 +++ php5-200409011830/main/streams/streams.c 2004-09-01 22:06:11.579747624 +0200 @@ -1276,7 +1276,12 @@ } } - if (php_stream_mmap_possible(src)) { + /* + ** We shouldn't use memory mapping on big files just for copying them. + ** It's not an efficient sollution for the vm. So for files above + ** 32mb (32*1024^2=33554432) we want a normal copy + */ + if ((php_stream_mmap_possible(src)) && (ssbuf.sb.st_size < 33554432)) { char *p; size_t mapped; ---------------------- ------------------------------------------------------------------------ [2004-09-01 21:55:01] info at pion dot xs4all dot nl I have checked again but i can't find the bug report. I have tested with "5.1.0-dev" with checked out version 200409011830 but the problem is still there. ------------------------------------------------------------------------ The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at http://bugs.php.net/29922 -- Edit this bug report at http://bugs.php.net/?id=29922&edit=1