Bug #71375 [Opn->Nab]: Memory not released from token_get_all()

From: Date: Fri, 15 Jan 2016 04:40:34 +0000
Subject: Bug #71375 [Opn->Nab]: Memory not released from token_get_all()
References: 1  Groups: php.bugs 
Request: Send a blank email to php-bugs+get-198664@lists.php.net to get a copy of this message
Edit report at https://bugs.php.net/bug.php?id=71375&edit=1

 ID:                 71375
 Updated by:         bwoebi@php.net
 Reported by:        pete at peteward dot net
 Summary:            Memory not released from token_get_all()
-Status:             Open
+Status:             Not a bug
 Type:               Bug
 Package:            Performance problem
 Operating System:   Centos Linux 7.0
 PHP Version:        7.0.2
 Block user comment: N
 Private report:     N

 New Comment:

This isn't a bug, but a side effect of the new memory manager which caches more memory and
doesn't release memory back to the OS so quickly.
Try setting the first parameter of memory_get_usage to false. You'll then see the memory PHP
really needs.

Also, try calling gc_mem_caches() first before measuring. This releases unused memory segments.

This is intentional behavior, though it is possible that this gets quite extreme in cases like
token_get_all().

See also bug #70098


Previous Comments:
------------------------------------------------------------------------
[2016-01-14 22:11:39] pete at peteward dot net

Description:
------------
Memory leaked from token_get_all() is much worse in php 7.x than in 5.x.

I have a large Symfony app with a container file string which when passed through token_get_all()
hangs on to a large amount of it's consumed memory, despite the peak consumption being less.

I haven't managed to identify which tokens are causing this as a basic very large string
didn't have this issue, however the test script below shows how it works with the
'real' content (a symfony container).



Test script:
---------------
<?php

printMemory();
$string = file_get_contents('http://dev.radiancedaily.com/appDevDebugProjectContainer.txt');
printMemory();
getAllTokensInScope($string);
printMemory();

function printMemory() {
    $mem = str_pad(round((memory_get_usage(true)/1024/1024)), 3, " ", STR_PAD_LEFT) .
"MB " .
           str_pad(round((memory_get_peak_usage(true)/1024/1024)), 3, " ", STR_PAD_LEFT) .
"MB ";
    print $mem . PHP_EOL;
}

function getAllTokensInScope($string) {
    $something = token_get_all($string);
}

Expected result:
----------------
Memory should be released outside of the scope of the function from which it's called.

PHP 5.4 and 5.6:

$ php -v
PHP 5.6.14-0+deb8u1 (cli) (built: Oct  4 2015 16:13:10) 
Copyright (c) 1997-2015 The PHP Group

$ php test.php 
  0MB   0MB 
  3MB   3MB 
 11MB 210MB 


Actual result:
--------------
Most of the memory is not released.

$ php -v
PHP 7.0.2 (cli) (built: Jan  9 2016 14:00:11) ( NTS )
Copyright (c) 1997-2015 The PHP Group

$ php test.php 
  2MB   2MB 
  7MB   7MB 
111MB 130MB 




------------------------------------------------------------------------



--
Edit this bug report at https://bugs.php.net/bug.php?id=71375&edit=1


Thread (3 messages)

« previous php.bugs (#198664) next »