Bug #77879 [Nab]: Usable memory

Edit report at https://bugs.php.net/bug.php?id=77879&edit=1 ID: 77879 User updated by: xxalfa at gmail dot com Reported by: xxalfa at gmail dot com Summary: Usable memory Status: Not a bug Type: Bug Package: Testing related Operating System: Windows PHP Version: 7.3.4 Block user comment: N Private report: N New Comment: Reply to requinix@php.net I am glad of every answer, only we both do not speak the same language. PHP can allocate as much memory as it needs. The amount of data I want to process requires knowing the amount of memory consumed to initiate paging operations. Any write access to the hard drive takes time, so I want to use as much memory as possible. Reply to nikic@php.net I thank you for your answer, which I now understand the way PHP works. I hope you can briefly answer my second question? When I open the task manager, I can keep track of how much memory a script consumes. Although PHP reserves full RAM, I can only fill it up to 40% at 16 MB. If I set the limit to 41% and fill the memory, I get a fatal error. In the Task Manager, a value of about 12 MB is given, why does not "memory_get_usage" provide the true consumption of 12 MB? Previous Comments: ------------------------------------------------------------------------ [2019-04-12 07:48:07] nikic@php.net Memory allocations are restricted to certain sizes for performance reasons. Small allocations are rounded up the next largest bin size. When the memory is reallocated with a size that still fits within the same bin, the reallocation is performed in-place and the reported memory usage will not change. ------------------------------------------------------------------------ [2019-04-12 03:06:55] requinix@php.net No, the memory does not increase by 1024 each time. It increases by whatever amount PHP allocates in order to expand the string by another 1024 bytes. As your output shows, the last step it went up by 4096 - enough for approximately 4 increases. Should it have only allocated 1k? Is it bad that PHP is reserving more memory than the precise amount it needs at that given moment? ------------------------------------------------------------------------ [2019-04-12 02:58:35] xxalfa at gmail dot com The script executes successfully because it has detected because the level of memory has not changed. However, the level increases by 1024 per cycle, if the function were to work properly, the script would continue to work. That's why it's a bug. ------------------------------------------------------------------------ [2019-04-12 02:26:32] requinix@php.net Your script is quitting because PHP didn't need to allocate more memory to handle another 1k characters. Why is that a bug? ------------------------------------------------------------------------ [2019-04-12 01:48:30] xxalfa at gmail dot com Description: ------------ The following script fills the memory with pointless data. Now the problem: From one point, the function "memory_get_usage" returns the same value, although the level has changed. This is important if you want to process data and save it only when a certain number of results have been generated. The second problem: Why are only 40% usable? Test script: --------------- <?php // C:\Users\User\Desktop\php-on-limit.cmd // @echo off // title PHP Development Server // cd "%cd%" // rem "C:\php\php.exe" "php-on-limit.php" // "C:\php\php.exe" "-d memory_limit=16M" "php-on-limit.php" // pause // C:\Users\User\Desktop\php-on-limit.php [PHP:7.3.3][PID:6336] //------------------------------------------------- // HEAD //------------------------------------------------- declare( strict_types = 1 ); header( 'Content-Type:text/plain' ); error_reporting( E_ALL ); ini_set( 'display_errors', '1' ); ini_set( 'html_errors', '0' ); define( 'CORE_DIR', dirname( __FILE__ ) . DIRECTORY_SEPARATOR ); isset( $argv ) or trigger_error( 'This is a command terminal application.', E_USER_ERROR ); echo __FILE__ . ' [PHP:' . phpversion() . '][PID:' . getmypid() . ']' . PHP_EOL . PHP_EOL; //------------------------------------------------- // CODE //------------------------------------------------- $measuring_point_of_time = microtime( true ); $memory_limit = conversion_back_to_bytes( ini_get( 'memory_limit' ) ); echo 'memory_limit -- ' . human_readable_file_size( $memory_limit ) . ' -- defined limit' . PHP_EOL; // $memory_limit *= 0.48; // 48% of 128M can only be used // $memory_limit *= 0.47; // 47% of 64M can only be used $memory_limit *= 0.40; // 40% of 16M can only be used // If the 40% is exceeded, it comes to a fatal error, which is not desirable. // Fatal error: Allowed memory size of 16 MByte exhausted (tried to allocate 6 MByte). echo 'memory_limit -- ' . human_readable_file_size( $memory_limit ) . ' -- usable limit' . PHP_EOL; echo 'memory_allocated -- ' . human_readable_file_size( memory_get_usage( true ) ) . ' -- memory_used -- ' . memory_get_usage() . PHP_EOL; $something_has_to_be_calculated = true; $results_of_the_calculations = null; $last_memory_usage = memory_get_usage(); while ( $something_has_to_be_calculated ) { $results_of_the_calculations .= str_repeat( '1', 1024 ); $currently_memory_usage = memory_get_usage(); if ( $currently_memory_usage === $last_memory_usage ) { echo PHP_EOL . 'Process recording: Memory function has not changed. currently_memory_usage and last_memory_usage are ' . $currently_memory_usage . PHP_EOL; break; } // echo chr( 0xD ); echo 'memory_allocated -- ' . human_readable_file_size( memory_get_usage( true ) ) . ' -- memory_used -- ' . memory_get_usage() . ' -- data_length -- ' . strlen( $results_of_the_calculations ); echo PHP_EOL; if ( memory_get_usage() >= 20 * 1024 * 1024 ) { echo PHP_EOL . 'Process recording: Normal break.' . PHP_EOL; break; } if ( memory_get_usage() >= $memory_limit ) { echo PHP_EOL . 'Process recording: Memory limit reached. Save calculated results so far.' . PHP_EOL; $results_of_the_calculations = null; } $last_memory_usage = memory_get_usage(); } echo PHP_EOL; $results_of_the_calculations = null; echo 'Process recording: Calculations completed. Results are saved.' . PHP_EOL . PHP_EOL; echo 'Processing time: ' . elapsed_time( $measuring_point_of_time ) . ' seconds.' . PHP_EOL . PHP_EOL; //------------------------------------------------- // FUNCTIONS //------------------------------------------------- function elapsed_time( $measuring_point_of_time ) { return number_format( microtime( true ) - $measuring_point_of_time, 3 ); } function human_readable_file_size( $file_size, $precision = 2 ) { $label = array( 'Bytes', 'kByte', 'MByte', 'GByte', 'TByte', 'PByte', 'EByte', 'ZByte', 'YByte' ); return $file_size ? round( $file_size / pow( 1024, ( $index = floor( log( $file_size, 1024 ) ) ) ), $precision ) . ' ' . $label[ $index ] : '0 Bytes'; } function conversion_back_to_bytes( $value ) { $unit = strtolower( substr( $value, -1 ) ); if ( $unit == 'k' ): return (integer) $value * 1024; endif; if ( $unit == 'm' ): return (integer) $value * 1048576; endif; if ( $unit == 'g' ): return (integer) $value * 1073741824; endif; return $value; } ?> Expected result: ---------------- The "memory_get_usage" function must continually display the current memory usage to effectively operate at the memory limit. Actual result: -------------- C:\Users\User\Desktop\php-on-limit.php [PHP:7.3.4][PID:1224] memory_limit -- 16 MByte -- defined limit memory_limit -- 6.4 MByte -- usable limit memory_allocated -- 2 MByte -- memory_used -- 410408 memory_allocated -- 2 MByte -- memory_used -- 411688 -- data_length -- 1024 memory_allocated -- 2 MByte -- memory_used -- 412968 -- data_length -- 2048 memory_allocated -- 2 MByte -- memory_used -- 414504 -- data_length -- 3072 memory_allocated -- 2 MByte -- memory_used -- 418600 -- data_length -- 4096 Process recording: Memory function has not changed. currently_memory_usage and last_memory_usage are 418520 Process recording: Calculations completed. Results are saved. Processing time: 0.001 seconds. ------------------------------------------------------------------------ -- Edit this bug report at https://bugs.php.net/bug.php?id=77879&edit=1