note 128410 added to function.inflate-add

From: Date: Thu, 20 Apr 2023 12:30:36 +0000
Subject: note 128410 added to function.inflate-add
Groups: php.notes 
Request: Send a blank email to php-notes+get-220365@lists.php.net to get a copy of this message
It's not obvious how to use this for _incremental_ decompression: You feed _the compressed data_ into inflate_add() _in pieces_. The internal state of the zlib context will make sure than you can split at any point and still get the correct total data out, as long as you keep reading until the end. In this way, you don't have to hold the complete uncompressed data in memory at any one time (and don't have to materialize it either as a file for gzopen() etc.), allowing you to parse files much bigger than the available php memory limit. <?php /* a good step size depends on the input's level of compression, unfortunately there's no obvious way to know that beforehand; in doubt instead choose a rather small value and glue the pieces together, until there's enough data for processing */ $step = 500000; $dataGz = load_gzip_compressed_data_to_string(); $start = 0; $outLen = 0; $ctxt = inflate_init(ZLIB_ENCODING_GZIP); $status = inflate_get_status($inflCtxt); while($status == ZLIB_OK) { $split = substr($dataGz, $start, $step); $dataFragment = inflate_add($inflCtxt, $split); /* process fragment, potentially keep parts across iterations */ $outLen += strlen($dataFragment); $status = inflate_get_status($inflCtxt); $start += $step; } echo 'Input: ' . strlen($dataGz) . ' Bytes / Output: ' . $outLen . ' Bytes.'; ?> N.B.: Archives of extremely high compression will still bomb out with a stupid and unnecessary memory exhaustion, as it's not possible to define a limit in inflate_init() similar to gzuncompress(). ---- Server IP: 45.112.84.4 Probable Submitter: 80.90.13.130 (proxied: 2a02:3037:605:eec5:388f:f66a:37fa:7936) ---- Manual Page -- https://php.net/manual/en/function.inflate-add.php Edit -- https://main.php.net/note/edit/128410 Del: integrated -- https://main.php.net/note/delete/128410/integrated Del: useless -- https://main.php.net/note/delete/128410/useless Del: bad code -- https://main.php.net/note/delete/128410/bad+code Del: spam -- https://main.php.net/note/delete/128410/spam Del: non-english -- https://main.php.net/note/delete/128410/non-english Del: in docs -- https://main.php.net/note/delete/128410/in+docs Del: other reasons-- https://main.php.net/note/delete/128410 Reject -- https://main.php.net/note/reject/128410 Search -- https://main.php.net/manage/user-notes.php

« previous php.notes (#220365) next »