Bug #81092 [Ver]: Calling fflush before stream_filter_remove results in corrupted stream

Edit report at https://bugs.php.net/bug.php?id=81092&edit=1 ID: 81092 Updated by: cmb@php.net Reported by: mark at klb dot jp Summary: Calling fflush before stream_filter_remove results in corrupted stream Status: Verified Type: Bug Package: Bzip2 Related Operating System: Linux PHP Version: 7.4 Assigned To: cmb Block user comment: N Private report: N New Comment: Argh, the erratic behavior on Windows was caused by shell_exec() which uses text mode[1]. I've used a file for the regression test; that fails without the patch, and works on Windows as well. [1] <https://www.php.net/shell_exec> Previous Comments: ------------------------------------------------------------------------ [2021-06-07 14:05:02] cmb@php.net The following pull request has been associated: Patch Name: Fix #81092: fflush before stream_filter_remove corrupts stream On GitHub: https://github.com/php/php-src/pull/7113 Patch: https://github.com/php/php-src/pull/7113.patch ------------------------------------------------------------------------ [2021-06-04 17:33:19] cmb@php.net My first assessment of this bug report was done on Windows, where I got more strange results, so I switched to Linux, where it quickly turned out that this regression has been introduced by the fix for bug #75776 (available as of PHP 7.4.17), and that there are no issues with zlib.deflate. > random_bytes() generate a string with a lot of entropy, which > will be difficult to compress and will result in larger compressed > block(s). Indeed, that's the issue. I'll investigate the Windows issues, and file bug reports if appropriate, but lets keep these out of this ticket. ------------------------------------------------------------------------ [2021-06-03 16:50:20] mark at klb dot jp Somehow this issue does not happen for me with php 7.4, and we started to have that issue as we upgraded from 7.4 to 8.0.6, so I assumed it was a php8 issue. With: $ php -v PHP 7.4.14 (cli) (built: Mar 6 2021 04:12:14) ( NTS ) Copyright (c) The PHP Group Zend Engine v3.4.0, Copyright (c) Zend Technologies with Zend OPcache v7.4.14, Copyright (c), by Zend Technologies (php 7.4.14 from Linux Gentoo, so might contain some patches) $ php bug.php int(8192) This seems to be related to large output of compressed data, so outputting something that compresses well (str_repeat, etc) will not cause this issue. random_bytes() generate a string with a lot of entropy, which will be difficult to compress and will result in larger compressed block(s). ------------------------------------------------------------------------ [2021-06-03 12:42:09] cmb@php.net I can confirm that the script outputs int(0), but I get the same result with PHP-7.4 either (regardless of explicitly calling fflush()). Interestingly, using a str_repeat("*", 8192) instead of random_bytes(8192) gives the expected result. If I use the zlib.deflate and gzinflate(), I get an explicit: Warning: gzinflate(): data error So apparently a general stream (compression) filter issue. ------------------------------------------------------------------------ [2021-05-31 23:17:35] mark at klb dot jp A couple notes: I am not sure this is a bz2 issue, this could come from filters too. Outputting to php://output using a filter and fflush()ing before removing the filter are conditions for this bug to happen. The bz2 stream generated is cut partially, using bzcat will for example output: php bug.php output | bzcat -tvv (stdin): [1: huff+mtf file ends unexpectedly ------------------------------------------------------------------------ The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at https://bugs.php.net/bug.php?id=81092 -- Edit this bug report at https://bugs.php.net/bug.php?id=81092&edit=1