Edit report at https://bugs.php.net/bug.php?id=70098&edit=1
ID: 70098
Updated by: dmitry@php.net
Reported by: robin dot kunde at recoursive dot com
Summary: Real memory usage doesn't decrease
Status: Assigned
Type: Bug
Package: Performance problem
Operating System: Ubuntu 15.04, OSX 10.10.4
PHP Version: 7.0.0beta1
Assigned To: dmitry
Block user comment: N
Private report: N
New Comment:
New memory manager may keep pages, previously allocated to serve some "small" blocks (less
than 3Kb), even if all of them were deallocated. These "small" blocks are cached (in
linked list) and may be reused later.
This strategy works well for real-life apps, because all pages are reclaimed on request shut-down
anyway. However, I agree, this may be not good enough for some long-running apps.
To fix this, we will need some kind of GC, that will traverse cache lists for each "small"
size and determine pages where all blocks are cached. Then this pages may be freed and all the
nested blocks removed from cache lists. This GC may be triggered when memory limit is reached, or
before allocating each new 2M chunk.
Previous Comments:
------------------------------------------------------------------------
[2015-07-25 07:11:28] alex at alex-at dot ru
Not only on OOM probably, but on forced gc_collect_cycles() call as well, to make forced cleaning
possible for long running scripts.
------------------------------------------------------------------------
[2015-07-21 15:02:13] rasmus@php.net
I think the idea of freeing these on an OOM is a good one. I would hate to see additional overhead
added to try to keep track of these for the common case of short-lived requests.
------------------------------------------------------------------------
[2015-07-21 03:19:17] laruence@php.net
pages are allocated together in trunk(~2M), which means if a page is not used anymore it won't
be able to release back to OS , unless the whole trunk is not used anymore..
------------------------------------------------------------------------
[2015-07-20 19:36:20] bwoebi@php.net
@laruence: That's right. But not the issue. Issue is rather that, having allocated a lot of
pages (e.g. during initialization) and then the variables all normally dtor'ed, the pages will
never be returned back to the OS for the whole lifetime of the script (which may be very long)...
Sure, it's rather an edge-case case, but I'd like to see completely unused pages released
back to the system
But it might involve a per-page counter and steal like a four instructions and a branch per
alloc/free⦠Not sure how much we can afford.
------------------------------------------------------------------------
[2015-07-20 13:28:32] robin dot kunde at recoursive dot com
Base on what you said, I performed another test:
I set memory_limit to 512M. At the end of the script I posted, after nulling out all the vars, and
memory usage being (Real: 422.00, Malloc: 0.36), I tried to read a 200M file. This triggered the
memory exhaustion fatal error. If I read the large file at the beginning of the script, and then
null the var, the script runs as expected and memory usage doesn't change much since it's
a large allocation.
I'm glad that the small bucket pool is counted against the memory limit and scripts can't
accidentally use up to twice as much memory. However, it does make the behavior of long running,
memory intensive script potentially unpredictable.
I understand the performance implications of not having to free those buckets. As a compromise, the
small bucket pool could be cleaned only when absolutely necessary.
------------------------------------------------------------------------
The remainder of the comments for this report are too long. To view
the rest of the comments, please view the bug report online at
https://bugs.php.net/bug.php?id=70098
--
Edit this bug report at https://bugs.php.net/bug.php?id=70098&edit=1