Bug #81659 [Opn->Ana]: stream_get_contents() loading entire file into memory
Edit report at https://bugs.php.net/bug.php?id=81659&edit=1
ID: 81659
Updated by: requinix@php.net
Reported by: terence-marks at zencontrol dot com
Summary: stream_get_contents() loading entire file into
memory
-Status: Open
+Status: Analyzed
Type: Bug
Package: Streams related
Operating System: Debian Buster
PHP Version: 8.0.13
Block user comment: N
Private report: N
New Comment:
PHP doesn't actually read the whole file, but will attempt to allocate enough memory to hold
the whole thing: there's a small optimization[1] that recognizes it knows the length of the
stream and so can make an educated guess as to how much it might need to read to get everything,
however that isn't taking into account that the current position in the stream can reduce that
number by quite a bit.
[1] https://github.com/php/php-src/blob/PHP-8.0.13/main/streams/streams.c#L1493
Previous Comments:
------------------------------------------------------------------------
[2021-11-26 04:17:25] terence-marks at zencontrol dot com
Description:
------------
Passing length of -1 to stream_get_contents() with the file position indicator of resource near the
end of file still loads the entire file into memory. Passing a fixed length to stream_get_contents()
does not produce the same behaviour.
Note: when running test script set memory_limit=64M
Test script:
---------------
<?php
$resource = fopen('/tmp/file1', 'a+');
for ($i = 0; $i < 64; ++$i)
{
$data = random_bytes(1024 * 1024);
$position = ftell($resource);
fwrite($resource, $data);
fseek($resource, $position);
stream_get_contents($resource, strlen($data));
}
$resource = fopen('/tmp/file2', 'a+');
for ($i = 0; $i < 64; ++$i)
{
$data = random_bytes(1024 * 1024);
$position = ftell($resource);
fwrite($resource, $data);
fseek($resource, $position);
stream_get_contents($resource, -1);
}
Actual result:
--------------
PHP Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to allocate 62922784 bytes)
in script.php on line 22
------------------------------------------------------------------------
--
Edit this bug report at https://bugs.php.net/bug.php?id=81659&edit=1
Thread (4 messages)