Bug #81092 [Ver]: Calling fflush before stream_filter_remove results in corrupted stream

Edit report at https://bugs.php.net/bug.php?id=81092&edit=1 ID: 81092 User updated by: mark at klb dot jp Reported by: mark at klb dot jp Summary: Calling fflush before stream_filter_remove results in corrupted stream Status: Verified Type: Bug Package: Streams related Operating System: Linux PHP Version: 7.4 Assigned To: cmb Block user comment: N Private report: N New Comment: Somehow this issue does not happen for me with php 7.4, and we started to have that issue as we upgraded from 7.4 to 8.0.6, so I assumed it was a php8 issue. With: $ php -v PHP 7.4.14 (cli) (built: Mar 6 2021 04:12:14) ( NTS ) Copyright (c) The PHP Group Zend Engine v3.4.0, Copyright (c) Zend Technologies with Zend OPcache v7.4.14, Copyright (c), by Zend Technologies (php 7.4.14 from Linux Gentoo, so might contain some patches) $ php bug.php int(8192) This seems to be related to large output of compressed data, so outputting something that compresses well (str_repeat, etc) will not cause this issue. random_bytes() generate a string with a lot of entropy, which will be difficult to compress and will result in larger compressed block(s). Previous Comments: ------------------------------------------------------------------------ [2021-06-03 12:42:09] cmb@php.net I can confirm that the script outputs int(0), but I get the same result with PHP-7.4 either (regardless of explicitly calling fflush()). Interestingly, using a str_repeat("*", 8192) instead of random_bytes(8192) gives the expected result. If I use the zlib.deflate and gzinflate(), I get an explicit: Warning: gzinflate(): data error So apparently a general stream (compression) filter issue. ------------------------------------------------------------------------ [2021-05-31 23:17:35] mark at klb dot jp A couple notes: I am not sure this is a bz2 issue, this could come from filters too. Outputting to php://output using a filter and fflush()ing before removing the filter are conditions for this bug to happen. The bz2 stream generated is cut partially, using bzcat will for example output: php bug.php output | bzcat -tvv (stdin): [1: huff+mtf file ends unexpectedly ------------------------------------------------------------------------ [2021-05-31 18:34:32] mark at klb dot jp Description: ------------ When using filters to output bzip2 compressed data, one may be tempted to call fflush() before stream_filter_remove() to ensure all data has been written. This used to work fine, however since php8, this causes the generated stream to be corrupt. Test script: --------------- <?php if ($_SERVER['argv'][1]??'' == 'output') { $stream = fopen('php://output', 'wb+'); $filter = stream_filter_append($stream, 'bzip2.compress', STREAM_FILTER_WRITE, ['blocks' => 9, 'work' => 0]); fwrite($stream, random_bytes(8192)); fflush($stream); stream_filter_remove($filter); // should call dtor exit; } $script = $_SERVER['argv'][0]; $res = shell_exec('php '.escapeshellarg($script).' output'); // should output bz encoded data $data = bzdecompress($res); var_dump(strlen($data)); Expected result: ---------------- int(8192) Actual result: -------------- int(0) ------------------------------------------------------------------------ -- Edit this bug report at https://bugs.php.net/bug.php?id=81092&edit=1