Bug #78987 [Com]: Memory problems running finfo::buffer with PHP_CLI

Edit report at https://bugs.php.net/bug.php?id=78987&edit=1 ID: 78987 Comment by: sydneymylee62 at gmail dot com Reported by: jhhillie at amazon dot com Summary: Memory problems running finfo::buffer with PHP_CLI Status: Verified Type: Bug Package: Filesystem function related Operating System: Ubuntu PHP Version: 7.4Git-2019-12-18 (Git) Block user comment: N Private report: N New Comment: Thanks for the update and quick reply. I'll be sure to keep an eye on this thread. Looking for the same issue. Bumped into your thread. Thanks for creating it. Looking forward for solution. https://www.tellpopeyes.buzz/ Previous Comments: ------------------------------------------------------------------------ [2020-06-12 07:12:11] magnar at myrtveit dot com I've created two scripts that illustrate the problem with finfo_buffer: Memory usage of finfo_file: https://3v4l.org/M3YLG Memory usage of finfo_buffer: https://3v4l.org/LNNoV On a 16M file, finfo_file uses 20M memory. finfo_buffer, however, uses 244M! ------------------------------------------------------------------------ [2020-02-13 08:15:27] nikic@php.net Related To: Bug #79263 ------------------------------------------------------------------------ [2019-12-18 10:58:00] nikic@php.net I can reproduce the large allocation. It is caused by https://github.com/php/php-src/blob/aadd5e69e01728adb78d03026beb9a9a0c7e75ef/ext/fileinfo/libmagic/encoding.c#L92. Basically, libmagic tries to detect the encoding of the file and does this by ... allocating a buffer of unicode characters for the whole file. For some reason this also uses "unsigned long" instead of "uint32_t" for each character, which means the allocation is not just 4 times as large, but 8 times as large as the original file! Current upstream version still looks about the same: https://github.com/file/file/blob/master/src/encoding.c Ideally this issue would be reported upstream and fixed there first. ------------------------------------------------------------------------ [2019-12-18 04:51:12] requinix@php.net > Failed identify data 12:cannot allocate 2516582408 bytes (Cannot allocate > memory)application/octet-stream According to that, libmagic is the one failing. Working for me with PHP 7.0-7.4 on Ubuntu. Does it actually include "application/octet-stream" in there? That is the correct MIME type... I assume the system is, in fact, running out of memory? What version of libmagic? ------------------------------------------------------------------------ [2019-12-18 04:12:28] jhhillie at amazon dot com Description: ------------ When calling finfo::buffer() with a crafted string, PHP tries to allocate an insane amount of memory to do so. I found a similar bug here - https://bugs.php.net/bug.php?id=68819 I suspect the bug may have been re-introduced at some stage during newer versions. 1) I tested it on the version my customer had been using - v7.0.33 (doesn't work) 2) I then tested on the latest php version - v7.4.0 (doesn't work) 3) Testing on an older version - v5.4.16 it works perfectly. Test script: --------------- Generated a random file of 300MB using: $ sudo fallocate -l 300M test-file The script used: <?php $content = file_get_contents('test-file', true); $fileInfo = new \finfo(FILEINFO_MIME_TYPE); var_dump($fileInfo->buffer($content)); ?> Expected result: ---------------- To provide the file metadata (I know there are other ways to do this): string(19) "application/x-empty" Actual result: -------------- PHP Warning: finfo::buffer(): Failed identify data 12:cannot allocate 2516582408 bytes (Cannot allocate memory)application/octet-stream in php shell code on line 1 bool(false) ------------------------------------------------------------------------ -- Edit this bug report at https://bugs.php.net/bug.php?id=78987&edit=1