Question about buffer size calculation in _php_stream_copy_to_mem_ex
| From: | Arvind Srinivasan | Date: | Thu, 19 Nov 2009 12:42:35 +0000 |
| Subject: | Question about buffer size calculation in _php_stream_copy_to_mem_ex | ||
| Groups: | php.internals | ||
| Request: | Send a blank email to internals+get-46101@lists.php.net to get a copy of this message | ||
I see the following in _php_stream_copy_to_mem_ex in main/streams/streams.c
int step = CHUNK_SIZE;
int min_room = CHUNK_SIZE / 4;
(..stuff deleted..)
/* avoid many reallocs by allocating a good sized chunk to begin with, if
* we can. Note that the stream may be filtered, in which case the stat
* result may be inaccurate, as the filter may inflate or deflate the
* number of bytes that we can read. In order to avoid an upsize followed
* by a downsize of the buffer, overestimate by the step size (which is
* 2K). */
if (php_stream_stat(src, &ssbuf) == 0 && ssbuf.sb.st_size > 0) {
max_len = ssbuf.sb.st_size + step;
} else {
max_len = step;
}
According to the comment the intention is to overestimate by 2K, but
CHUNK_SIZE = 8192 and that leads quite a bit of over-allocation of
memory.
Is the following a better setting for max_len in the if block?
max_len = (((ssbuf.sb.st_size + min_room)/step) + 1) * step;
The following are sample values that show the current vs new size calculation.
ssbuf.sb.st_size, max_len (current), max_len (new)
( 100, 8292, 8192)
( 1000, 9192, 8192)
( 5000, 13192, 8192)
( 7000, 15192, 16384)
( 13000, 21192, 16384)
( 22000, 30192, 24576)
thanks,
arvi