Edit report at https://bugs.php.net/bug.php?id=51056&edit=1
ID: 51056
Updated by: kalle@php.net
Reported by: magicaltux@php.net
Summary: fread() on blocking stream will block even if data
is available
-Status: Analyzed
+Status: Assigned
Type: Bug
Package: Streams related
Operating System: Linux Gentoo 2.6.32
PHP Version: 5.5.0 alpha
Assigned To: cataphract
Block user comment: N
Private report: N
Previous Comments:
------------------------------------------------------------------------
[2013-01-06 09:01:59] gauthierm@php.net
Related To: Bug #52602
------------------------------------------------------------------------
[2010-09-28 02:07:19] cataphract@php.net
A small correction: it's not that "never reads from the socket more than one packet at a
time" as I and the manual say. It's that it does only one call to recv().
If we're blocking waiting for data and a packet arrives, then recv() will return only the
contents of that packet ("The receive calls normally return any data available, up to the
requested amount, rather than waiting for receipt of the full amount requested."). However, if
several packets have been received since the last call to fread, recv() will return the most data it
can, possibly several packets.
But this is a minor documentation issue and not very relevant in this discussion.
------------------------------------------------------------------------
[2010-09-28 01:55:15] cataphract@php.net
The bug reported in right on this issue.
As it stands, it is completely unpredictable whether an fread call will block.
The point of putting a stream into the readfs set of stream_select is to know whether a call to
fread will block or not. stream_select has an emultaion feature that returns the stream if the
stream buffer has data, even if there's no more data to be read on the socket. See bug #52602.
As it stands, if there's data in the stream buffer but not on the socket and the user asks for
more data than what's in the buffer, the call to fread will block, even though stream_select
returned the stream. The usual (and documented) semantics of select() would mean a call
wouldn't block:
«The streams listed in the read array will be watched to see if characters become available for
reading (more precisely, to see if a read will not block - in particular, a stream resource is also
ready on end-of-file, in which case an fread() will return a zero length string).»
Therefore, it's the current behavior that's unpredictable. We can never know whether a
call to fread will block.
The solution could be:
* Return whatever there's on the buffer, even if it's less than what was asked.
* Try to fill the rest of the buffer only with non-blocking reads.
* Call the real select (no emulation like in stream_select) and fill the rest of the buffer only
with the pending data.
It would be unwise, however, to have this behavior change introduced for non-sockets. The problem is
that many scripts admit that if they asked for n bytes, they will receive n bytes, except in the
last call.
For sockets, where the problem is more serious (since we never know when a packet will arrive, we
can block for a long time), this would be a minor BC break, because in that case fread never reads
from the socket more than one packet at a time and since the application can't know whether the
data it expects will arrive on one or several packets it already has to do deal with variable-length
fread returns. Indeed, the documentation for fread says:
«When reading from anything that is not a regular local file, such as streams returned when
reading remote files or from popen() and fsockopen(), reading will stop after a packet is available.
This means that you should collect the data together in chunks as shown in the examples below.»
------------------------------------------------------------------------
[2010-03-12 14:53:55] lbarnaud@php.net
I see your point in wanting read() behavior. Whether or not to implement fread() or read() one is
arguable. However the specific behavior you are asking for is not reliable for several reasons, and
IMHO (I may be wrong) you want this behavior for bad reasons. Let me explain this :
> By the way using nonblocking mode makes no sense with provided example. It would just make the
> program use 100% cpu.
This is why you don't want to use non-blocking streams. If you use stream_select() you will
never end up using 100% CPU : Your PHP process will only do an idle wait in stream_select() and
consume no CPU at all.
Example :
stream_set_blocking($stream, 0);
while (stream_select($r,$w,$e, $stream, $sec, $usec)) { /* block until data is available for read
and/or write in $stream. */
$data = fread($stream, 8192); /* read all available data, up to 8192 bytes. Returns only 1 byte if
only 1 byte is available and never blocks. */
}
> If end of email is reached while a read is in progress and a new read is called, it will block
> until the server closes connections
With your patch (or with the read behavior you want) it will still block. And it will block
randomly, in an unpredictable manner.
Please see the following example :
Say the buffer has 250 bytes in it.
fread(100) -> buffer.length-=100, buffer.length == 100
fread(100) -> buffer.length-=100, buffer.length == 50
fread(100) -> with your patch it would return the last 50 available bytes
Now this other example with a buffer with only 200 bytes in it :
Say the buffer has 200 bytes in it.
fread(100) -> buffer.length-=100, buffer.length == 100
fread(100) -> buffer.mength-=100, buffer.length == 0
fread(100) -> buffer is 0, this blocks, and you can't control this (you don't control
the buffer, and don't know anything about it in a php script)
Please see 51056-3.phpt.
With current behavior it will block too, but in a predictable maner.
------------------------------------------------------------------------
[2010-03-12 03:12:35] magicaltux@php.net
So, it is normal for php's fread() to return immediatly when less data than asked is available,
unless this data arrived while a previous call of fread() was done and there was
too much data ?
Let me just state that this doesn't makes sense.
I tested stdc's fread() and could confirm that its behaviour is consistent: it will only return
when it has collected the data it needed, when EOF is reached or when an error
occurs.
It seems that PHP's php_stream_read() is closer to read() syscall than to stdc's fread(),
except for this one specific behaviour.
> It follows fread() behavior since years and I believe it should not change.
I believe the problem comes from the new streams api which is an attempt to make the socket api
obsolete. In fact stream functions (including fread()) behave the same way the
old socket counterpart did when passed a socket.
The correct behaviour (as defined by common sense, and confirmed by PHP 4.4.9) :
Testing PHP version: 4.4.9
socket_read took 0.06ms to read 8 bytes
socket_read took 5.08ms to read 256 bytes
socket_read took 0.01ms to read 45 bytes
socket_read took 0.08ms to read 8 bytes
socket_read took 5.06ms to read 256 bytes
socket_read took 0.01ms to read 45 bytes
socket_read took 0.07ms to read 8 bytes
socket_read took 5.05ms to read 256 bytes
socket_read took 0.01ms to read 45 bytes
socket_read took 0.08ms to read 8 bytes
Testing with PHP 5.1.0 (first version containing stream_socket_pair()) exhibits a change of
behaviour due to the new stream api.
Both tests 51056.phpt and 51056-2.phpt pass on PHP 4.4.9.
By the way using nonblocking mode makes no sense with provided example. It would just make the
program use 100% cpu. For example a PHP program reading an email from a POP3
server might lockdown because of this bug in blocking mode. If end of email is reached while a read
is in progress and a new read is called, it will block until the server
closes connections (expected behaviour = return remaining data).
As a PHP sockets programmer (I believe my experience when it comes to php and sockets is not
negligeable) I say once more that *this* fread()'s behaviour is not consistent.
fread() in blocking mode should block until it has enough bytes or return as soon as some bytes are
avaialble. Blocking should not depend on when data has arrived.
------------------------------------------------------------------------
The remainder of the comments for this report are too long. To view
the rest of the comments, please view the bug report online at
https://bugs.php.net/bug.php?id=51056
--
Edit this bug report at https://bugs.php.net/bug.php?id=51056&edit=1