Doc #51056 [ReO->Ana]: fread() on blocking stream will block even if data is available

Edit report at https://bugs.php.net/bug.php?id=51056&edit=1 ID: 51056 Updated by: philip@php.net Reported by: magicaltux@php.net Summary: fread() on blocking stream will block even if data is available -Status: Re-Opened +Status: Analyzed Type: Documentation Problem -Package: Documentation problem +Package: Streams related Operating System: Linux Gentoo 2.6.32 -PHP Version: 5.3.1 +PHP Version: 5.5.0 alpha -Assigned To: +Assigned To: cataphract Block user comment: N Private report: N Previous Comments: ------------------------------------------------------------------------ [2013-01-06 09:01:59] gauthierm@php.net Related To: Bug #52602 ------------------------------------------------------------------------ [2010-09-28 02:07:19] cataphract@php.net A small correction: it's not that "never reads from the socket more than one packet at a time" as I and the manual say. It's that it does only one call to recv(). If we're blocking waiting for data and a packet arrives, then recv() will return only the contents of that packet ("The receive calls normally return any data available, up to the requested amount, rather than waiting for receipt of the full amount requested."). However, if several packets have been received since the last call to fread, recv() will return the most data it can, possibly several packets. But this is a minor documentation issue and not very relevant in this discussion. ------------------------------------------------------------------------ [2010-09-28 01:55:15] cataphract@php.net The bug reported in right on this issue. As it stands, it is completely unpredictable whether an fread call will block. The point of putting a stream into the readfs set of stream_select is to know whether a call to fread will block or not. stream_select has an emultaion feature that returns the stream if the stream buffer has data, even if there's no more data to be read on the socket. See bug #52602. As it stands, if there's data in the stream buffer but not on the socket and the user asks for more data than what's in the buffer, the call to fread will block, even though stream_select returned the stream. The usual (and documented) semantics of select() would mean a call wouldn't block: «The streams listed in the read array will be watched to see if characters become available for reading (more precisely, to see if a read will not block - in particular, a stream resource is also ready on end-of-file, in which case an fread() will return a zero length string).» Therefore, it's the current behavior that's unpredictable. We can never know whether a call to fread will block. The solution could be: * Return whatever there's on the buffer, even if it's less than what was asked. * Try to fill the rest of the buffer only with non-blocking reads. * Call the real select (no emulation like in stream_select) and fill the rest of the buffer only with the pending data. It would be unwise, however, to have this behavior change introduced for non-sockets. The problem is that many scripts admit that if they asked for n bytes, they will receive n bytes, except in the last call. For sockets, where the problem is more serious (since we never know when a packet will arrive, we can block for a long time), this would be a minor BC break, because in that case fread never reads from the socket more than one packet at a time and since the application can't know whether the data it expects will arrive on one or several packets it already has to do deal with variable-length fread returns. Indeed, the documentation for fread says: «When reading from anything that is not a regular local file, such as streams returned when reading remote files or from popen() and fsockopen(), reading will stop after a packet is available. This means that you should collect the data together in chunks as shown in the examples below.» ------------------------------------------------------------------------ [2010-03-12 14:53:55] lbarnaud@php.net I see your point in wanting read() behavior. Whether or not to implement fread() or read() one is arguable. However the specific behavior you are asking for is not reliable for several reasons, and IMHO (I may be wrong) you want this behavior for bad reasons. Let me explain this : > By the way using nonblocking mode makes no sense with provided example. It would just make the > program use 100% cpu. This is why you don't want to use non-blocking streams. If you use stream_select() you will never end up using 100% CPU : Your PHP process will only do an idle wait in stream_select() and consume no CPU at all. Example : stream_set_blocking($stream, 0); while (stream_select($r,$w,$e, $stream, $sec, $usec)) { /* block until data is available for read and/or write in $stream. */ $data = fread($stream, 8192); /* read all available data, up to 8192 bytes. Returns only 1 byte if only 1 byte is available and never blocks. */ } > If end of email is reached while a read is in progress and a new read is called, it will block > until the server closes connections With your patch (or with the read behavior you want) it will still block. And it will block randomly, in an unpredictable manner. Please see the following example : Say the buffer has 250 bytes in it. fread(100) -> buffer.length-=100, buffer.length == 100 fread(100) -> buffer.length-=100, buffer.length == 50 fread(100) -> with your patch it would return the last 50 available bytes Now this other example with a buffer with only 200 bytes in it : Say the buffer has 200 bytes in it. fread(100) -> buffer.length-=100, buffer.length == 100 fread(100) -> buffer.mength-=100, buffer.length == 0 fread(100) -> buffer is 0, this blocks, and you can't control this (you don't control the buffer, and don't know anything about it in a php script) Please see 51056-3.phpt. With current behavior it will block too, but in a predictable maner. ------------------------------------------------------------------------ [2010-03-12 03:12:35] magicaltux@php.net So, it is normal for php's fread() to return immediatly when less data than asked is available, unless this data arrived while a previous call of fread() was done and there was too much data ? Let me just state that this doesn't makes sense. I tested stdc's fread() and could confirm that its behaviour is consistent: it will only return when it has collected the data it needed, when EOF is reached or when an error occurs. It seems that PHP's php_stream_read() is closer to read() syscall than to stdc's fread(), except for this one specific behaviour. > It follows fread() behavior since years and I believe it should not change. I believe the problem comes from the new streams api which is an attempt to make the socket api obsolete. In fact stream functions (including fread()) behave the same way the old socket counterpart did when passed a socket. The correct behaviour (as defined by common sense, and confirmed by PHP 4.4.9) : Testing PHP version: 4.4.9 socket_read took 0.06ms to read 8 bytes socket_read took 5.08ms to read 256 bytes socket_read took 0.01ms to read 45 bytes socket_read took 0.08ms to read 8 bytes socket_read took 5.06ms to read 256 bytes socket_read took 0.01ms to read 45 bytes socket_read took 0.07ms to read 8 bytes socket_read took 5.05ms to read 256 bytes socket_read took 0.01ms to read 45 bytes socket_read took 0.08ms to read 8 bytes Testing with PHP 5.1.0 (first version containing stream_socket_pair()) exhibits a change of behaviour due to the new stream api. Both tests 51056.phpt and 51056-2.phpt pass on PHP 4.4.9. By the way using nonblocking mode makes no sense with provided example. It would just make the program use 100% cpu. For example a PHP program reading an email from a POP3 server might lockdown because of this bug in blocking mode. If end of email is reached while a read is in progress and a new read is called, it will block until the server closes connections (expected behaviour = return remaining data). As a PHP sockets programmer (I believe my experience when it comes to php and sockets is not negligeable) I say once more that *this* fread()'s behaviour is not consistent. fread() in blocking mode should block until it has enough bytes or return as soon as some bytes are avaialble. Blocking should not depend on when data has arrived. ------------------------------------------------------------------------ The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at https://bugs.php.net/bug.php?id=51056 -- Edit this bug report at https://bugs.php.net/bug.php?id=51056&edit=1