Req #73887 [Com]: Improve realpath cache eviction strategy

Edit report at https://bugs.php.net/bug.php?id=73887&edit=1 ID: 73887 Comment by: spam2 at rhsoft dot net Reported by: maggus dot staab at gmail dot com Summary: Improve realpath cache eviction strategy Status: Open Type: Feature/Change Request Package: Performance problem PHP Version: any Block user comment: N Private report: N New Comment: anyone considered the design flaw described in https://bugs.php.net/bug.php?id=73888 get fixed? re-base the patch for current master to enable it at all beause you guys think it's amrt hardcode that two times instead give the admin a option or at least compile-option to state "we don't allow link/symlink anyways so the cache is safe even with open_basedir" was a reminder taht it has a ton of problems /* Disable realpath cache if an open_basedir is set */ /*if (PG(open_basedir) && *PG(open_basedir)) { */ /* CWDG(realpath_cache_size_limit) = 0; */ /*}*/ Previous Comments: ------------------------------------------------------------------------ [2017-07-26 11:15:19] spam2 at rhsoft dot net someone pointed out offlist that this maybe is not clear: put a file on a large and busy server with that line <?php echo count(realpath_cache_get());?> call that URL 4 times and you get that numbers below 325 6 57 1692 you can hardly tell me to get a useful hitrate with such randomness all over workers ------------------------------------------------------------------------ [2017-07-26 10:56:10] spam2 at rhsoft dot net which performance do we really have? please read the bugreport i linked when i make 4 calls to a website and get that numbers below it's all said about the hitrate in real life and how high it could be - it needs a lot of requests to fill all that cache instances and after they are filled 'MaxRequestsPerChild' comes and kills the process, starts a new one and it's cache is empty again - so what *real world* performance are we talking about? clearstatcache(true, $path) is completly worthless because it resets the cache on one of propably many hundret workers which makes the whole cache dangerous while without that issue you could set "realpath_cache_ttl" to hours instead short values with 4 MB realpath_cache_size you have probably 1000*4 MB in case of 1000 workers, watsing 4 GB RAM, starting oom-killer or crashing the whole machine instead any benefit - so in summary the probably needed locks are the smallest problem at all echo count(realpath_cache_get()); 325 6 57 1692 ------------------------------------------------------------------------ [2017-07-26 10:26:43] maggus dot staab at gmail dot com adding a shared cache would also imply a locking/concurrency mechanism which could also be less performant and more complex in comparision what we have today. ------------------------------------------------------------------------ [2017-07-26 10:17:54] spam2 at rhsoft dot net first it would be much more important to have the cache shared between workers and so fix the split-brain and overhead of maintaining a cache for every process with a very bad hitrate on mahines with hundrets of workers see https://bugs.php.net/bug.php?id=73888 - "We recently changed the default realpath cache" is a big mistake as long as verey single worker has it's own cache with that size and clearstatcache() after change a file don't work at all with the current design ------------------------------------------------------------------------ [2017-01-07 13:45:40] maggus dot staab at gmail dot com Description: ------------ We recently changed the default realpath cache size as we realized that the default size is too small, see https://github.com/php/php-src/pull/2271. After a closer look into the realpath cache impl krakjoe and bwoebi mentioned, that cache eviction of this cache is very suboptiomal. Cache items are only evicted after a find. Entries never get evicted when no longer used. Php is forced todo some expensive file-io operations when the realpath cache cannot deliver the information from memory. This is especially expensive for long running php scripts, like cli based processes/webservers/... A first improvement would be some kind garbage collection, which isnt bound to a previous find. 2nd the cache could benefit from some kind of LRU cache. 3rd krakjoe mentioned that it might be sometimes usefull to share the cache information in shm. Related discussion: http://chat.stackoverflow.com/transcript/message/34952738#34952738 Starting 5:32 PM ------------------------------------------------------------------------ -- Edit this bug report at https://bugs.php.net/bug.php?id=73887&edit=1