Bug #73262 [NEW]: 8192 CHUNK_SIZE and PHP_SOCK_CHUNK_SIZE makes copy() slow

From: jeffreydwalter at gmail dot com Operating system: Irrelevant PHP version: Irrelevant Package: Performance problem Bug Type: Bug Bug description:8192 CHUNK_SIZE and PHP_SOCK_CHUNK_SIZE makes copy() slow Description: ------------ ***I have not tested this outside of FreeBSD 10.3, but it should be an issue on most if not all platforms*** copy() is several orders of magnitude slower than exec('cp ...') on the same partition. It is even worse copying across partitions. After digging through the source code, I discovered that it's because of the shear number of buffered write() calls that are made on the underlying socket in _php_stream_write_buffer(). The buffer's size is hardcoded to 8192 <a href="https://github.com/php/php-src/blob/PHP-5.6.20/main/streams/php_streams_int.h#L49">here</a> and <a href="https://github.com/php/php-src/blob/PHP-5.6.20/main/php_network.h#L210">here</a>. If I change CHUNK_SIZE and PHP_SOCK_CHUNK_SIZE to 8192*24, the php copy() function will perform at a comparable speed to exec('cp ...'). I don't fully understand the ramifications of changing that buffer outside of greater memory usage, and I haven't tested it on network-related stream operations. So if there's a better way I'd love to know. I do realize that I could create a stream context in php, set the buffer size, and pass that context to copy(), but that doesn't seem like a workable solution. Perhaps a better solution would be to make those CHUNK_SIZE defines tunable at compile time so they can be optimized for one's specific platform? Test script: --------------- <?php # Change this is you want to test with a different file size. define('FILE_SIZE_IN_MB', 250); printf('Generating %sMB dummy file.'.PHP_EOL, FILE_SIZE_IN_MB); echo shell_exec('dd if=/dev/urandom of=dummy.txt bs=1048576 count='.FILE_SIZE_IN_MB.' 2>&1 | grep -v records'); $copy_start = time(); copy('dummy.txt', 'copied_with_php_copy.txt'); $copy_done = time(); $cp_start = time(); exec('cp dummy.txt copied_with_cp.txt'); $cp_done = time(); echo 'copy() took: '.($copy_done-$copy_start).'s'.PHP_EOL; echo 'cp took: '.($cp_done-$cp_start).'s'.PHP_EOL; echo shell_exec('ls -al dummy.txt copied_with_php_copy.txt copied_with_cp.txt').PHP_EOL; echo 'Removing test files.'.PHP_EOL; exec('rm -rf dummy.txt copied_with_php_copy.txt copied_with_cp.txt'); Expected result: ---------------- php_stream_write took 0 seconds 796 milliseconds (I added this line in stream.c) php_stream_write() with mmap used (I added this line in stream.c) php output: Generating 250MB dummy file. copy() took: 1s cp took: 1s -rw-r--r-- 1 262144000 Oct 6 20:00 copied_with_cp.txt -rw-r--r-- 1 262144000 Oct 6 20:00 copied_with_php_copy.txt -rw-r--r-- 1 262144000 Oct 6 19:59 dummy.txt Removing test files. Actual result: -------------- php_stream_write took 5 seconds 562 milliseconds (I added this line in stream.c) php_stream_write() with mmap used (I added this line in stream.c) php output: php blah.php Generating 250MB dummy file. copy() took: 13s cp took: 0s -rw-r--r-- 1 262144000 Oct 6 20:00 copied_with_cp.txt -rw-r--r-- 1 262144000 Oct 6 20:00 copied_with_php_copy.txt -rw-r--r-- 1 262144000 Oct 6 19:59 dummy.txt Removing test files. -- Edit bug report at https://bugs.php.net/bug.php?id=73262&edit=1 -- Try a snapshot (PHP 5.4): https://bugs.php.net/fix.php?id=73262&r=trysnapshot54 Try a snapshot (PHP 5.5): https://bugs.php.net/fix.php?id=73262&r=trysnapshot55 Try a snapshot (trunk): https://bugs.php.net/fix.php?id=73262&r=trysnapshottrunk Fixed in SVN: https://bugs.php.net/fix.php?id=73262&r=fixed Fixed in release: https://bugs.php.net/fix.php?id=73262&r=alreadyfixed Need backtrace: https://bugs.php.net/fix.php?id=73262&r=needtrace Need Reproduce Script: https://bugs.php.net/fix.php?id=73262&r=needscript Try newer version: https://bugs.php.net/fix.php?id=73262&r=oldversion Not developer issue: https://bugs.php.net/fix.php?id=73262&r=support Expected behavior: https://bugs.php.net/fix.php?id=73262&r=notwrong Not enough info: https://bugs.php.net/fix.php?id=73262&r=notenoughinfo Submitted twice: https://bugs.php.net/fix.php?id=73262&r=submittedtwice register_globals: https://bugs.php.net/fix.php?id=73262&r=globals PHP 4 support discontinued: https://bugs.php.net/fix.php?id=73262&r=php4 Daylight Savings: https://bugs.php.net/fix.php?id=73262&r=dst IIS Stability: https://bugs.php.net/fix.php?id=73262&r=isapi Install GNU Sed: https://bugs.php.net/fix.php?id=73262&r=gnused Floating point limitations: https://bugs.php.net/fix.php?id=73262&r=float No Zend Extensions: https://bugs.php.net/fix.php?id=73262&r=nozend MySQL Configuration Error: https://bugs.php.net/fix.php?id=73262&r=mysqlcfg