Edit report at https://bugs.php.net/bug.php?id=38567&edit=1
ID: 38567
Updated by: nikic@php.net
Reported by: glen at delfi dot ee
Summary: file operations make loads of lstat64() calls and
making code ineffective
Status: Open
Type: Feature/Change Request
Package: *General Issues
PHP Version: 5.1.5
Block user comment: N
Private report: N
New Comment:
@rhsoft: That's not relevant to this issue.
From the strace it looks like we're still stat'ing all path components, even though the
full path is already not found. That does seem rather fishy.
Previous Comments:
------------------------------------------------------------------------
[2017-09-21 12:03:49] spam2 at rhsoft dot net
see https://bugs.php.net/bug.php?id=73888
increase it only wastes more memory for each worker which has it's on isolated view of the
cache and as requests are spreaded over workers you won't have cachehits most of the time until
all workers have everything cached
well, and in the real world workers are restarted after handling a few thousand requests and then it
needs to start caching from scratch again
------------------------------------------------------------------------
[2017-09-21 11:56:14] glen at delfi dot ee
i do not see any improvement, no matter what php version or cache size is.
post your result proving otherwise, maybe there's something hidden in plain sight.
â php70 -i|grep realpath_cache_size
realpath_cache_size => 4096K => 4096K
â php70 -d realpath_cache_size=128000k -i|grep realpath_cache_size
realpath_cache_size => 128000k => 128000k
â php70 -d realpath_cache_size=128000k --version
PHP 7.0.18 (cli) (built: Apr 24 2017 20:13:36) ( NTS )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.0.0, Copyright (c) 1998-2017 Zend Technologies
â strace php70 -d realpath_cache_size=128000k -r 'for ($i=0;$i<5;$i++)
{$fp=fopen("/home/glen/tmp/php/long/test/dir/is/here/$i","w");fclose($fp);}'
2>&1 |grep /home
lstat("/home/glen/.local/npm/bin/php70", 0x7fff98552520) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test/dir/is/here/0", 0x7fff98551160) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is/here", 0x7fff98550fc0) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is", 0x7fff98550e20) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test/dir", 0x7fff98550c80) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test", 0x7fff98550af0) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long", 0x7fff98550960) = -1 ENOENT (No such file or directory)
lstat("/home/glen/tmp/php", 0x7fff985507d0) = -1 ENOENT (No such file or directory)
lstat("/home/glen/tmp", {st_mode=S_IFDIR|0700, st_size=4096, ...}) = 0
lstat("/home/glen", {st_mode=S_IFDIR|0775, st_size=12288, ...}) = 0
lstat("/home", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
open("/home/glen/tmp/php/long/test/dir/is/here/0", O_WRONLY|O_CREAT|O_TRUNC, 0666) = -1
ENOENT (No such file or directory)
write(2, "PHP Warning: fopen(/home/glen/t"..., 145PHP Warning:
fopen(/home/glen/tmp/php/long/test/dir/is/here/0): failed to open stream: No such file or directory
in Command line code on line 1
lstat("/home/glen/tmp/php/long/test/dir/is/here/1", 0x7fff98551160) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is/here", 0x7fff98550fc0) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is", 0x7fff98550e20) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test/dir", 0x7fff98550c80) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test", 0x7fff98550af0) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long", 0x7fff98550960) = -1 ENOENT (No such file or directory)
lstat("/home/glen/tmp/php", 0x7fff985507d0) = -1 ENOENT (No such file or directory)
open("/home/glen/tmp/php/long/test/dir/is/here/1", O_WRONLY|O_CREAT|O_TRUNC, 0666) = -1
ENOENT (No such file or directory)
write(2, "PHP Warning: fopen(/home/glen/t"..., 145PHP Warning:
fopen(/home/glen/tmp/php/long/test/dir/is/here/1): failed to open stream: No such file or directory
in Command line code on line 1
lstat("/home/glen/tmp/php/long/test/dir/is/here/2", 0x7fff98551160) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is/here", 0x7fff98550fc0) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is", 0x7fff98550e20) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test/dir", 0x7fff98550c80) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test", 0x7fff98550af0) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long", 0x7fff98550960) = -1 ENOENT (No such file or directory)
lstat("/home/glen/tmp/php", 0x7fff985507d0) = -1 ENOENT (No such file or directory)
open("/home/glen/tmp/php/long/test/dir/is/here/2", O_WRONLY|O_CREAT|O_TRUNC, 0666) = -1
ENOENT (No such file or directory)
write(2, "PHP Warning: fopen(/home/glen/t"..., 145PHP Warning:
fopen(/home/glen/tmp/php/long/test/dir/is/here/2): failed to open stream: No such file or directory
in Command line code on line 1
lstat("/home/glen/tmp/php/long/test/dir/is/here/3", 0x7fff98551160) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is/here", 0x7fff98550fc0) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is", 0x7fff98550e20) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test/dir", 0x7fff98550c80) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test", 0x7fff98550af0) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long", 0x7fff98550960) = -1 ENOENT (No such file or directory)
lstat("/home/glen/tmp/php", 0x7fff985507d0) = -1 ENOENT (No such file or directory)
open("/home/glen/tmp/php/long/test/dir/is/here/3", O_WRONLY|O_CREAT|O_TRUNC, 0666) = -1
ENOENT (No such file or directory)
write(2, "PHP Warning: fopen(/home/glen/t"..., 145PHP Warning:
fopen(/home/glen/tmp/php/long/test/dir/is/here/3): failed to open stream: No such file or directory
in Command line code on line 1
lstat("/home/glen/tmp/php/long/test/dir/is/here/4", 0x7fff98551160) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is/here", 0x7fff98550fc0) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is", 0x7fff98550e20) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test/dir", 0x7fff98550c80) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test", 0x7fff98550af0) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long", 0x7fff98550960) = -1 ENOENT (No such file or directory)
lstat("/home/glen/tmp/php", 0x7fff985507d0) = -1 ENOENT (No such file or directory)
open("/home/glen/tmp/php/long/test/dir/is/here/4", O_WRONLY|O_CREAT|O_TRUNC, 0666) = -1
ENOENT (No such file or directory)
write(2, "PHP Warning: fopen(/home/glen/t"..., 145PHP Warning:
fopen(/home/glen/tmp/php/long/test/dir/is/here/4): failed to open stream: No such file or directory
in Command line code on line 1
â
------------------------------------------------------------------------
[2017-09-21 11:06:34] cmb@php.net
That appears to be related to the ini directive realpath_cache_size[1], which
defaults to only 16k before PHP 7.0.16. Try to increase the setting to 4096k,
for instance.
[1] <http://www.php.net/manual/en/ini.core.php#ini.realpath-cache-size>
------------------------------------------------------------------------
[2017-01-23 09:43:47] glen at delfi dot ee
â php --version
PHP 5.6.27 (cli) (built: Nov 21 2016 23:08:15)
Copyright (c) 1997-2016 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2016 Zend Technologies
with Zend OPcache v7.0.6-dev, Copyright (c) 1999-2016, by Zend Technologies
with Xdebug v2.4.1, Copyright (c) 2002-2016, by Derick Rethans
â php -r 'var_dump(ini_get("open_basedir"));'
Command line code:1:
string(0) ""
â strace php -r 'for ($i=0;$i<5;$i++)
{$fp=fopen("/home/glen/tmp/php/long/test/dir/is/here/$i","w");fclose($fp);}'
2>&1 |grep /home
open("/home/glen/.config/ImageMagick/log.xml", O_RDONLY) = -1 ENOENT (No such file or
directory)
open("/home/glen/.magick/log.xml", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/home/glen/.config/ImageMagick/policy.xml", O_RDONLY) = -1 ENOENT (No such file or
directory)
open("/home/glen/.magick/policy.xml", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/home/glen/.krb5/config", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/home/glen/.snmp", 0x7ffcda428cf0) = -1 ENOENT (No such file or directory)
stat("/home/glen/.snmp", 0x7ffcda428cf0) = -1 ENOENT (No such file or directory)
stat("/home/glen/.snmp/tls/ca-certs", 0x7ffcda426870) = -1 ENOENT (No such file or
directory)
stat("/home/glen/.snmp/tls/certs", 0x7ffcda426870) = -1 ENOENT (No such file or directory)
stat("/home/glen/.snmp/tls/private", 0x7ffcda426870) = -1 ENOENT (No such file or
directory)
open("/home/glen/.snmp/mibs", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = -1 ENOENT (No
such file or directory)
stat("/home/glen/.snmp", 0x7ffcda428d00) = -1 ENOENT (No such file or directory)
stat("/home/glen/.snmp", 0x7ffcda428d00) = -1 ENOENT (No such file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is/here/0", 0x7ffcda426c80) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is/here", 0x7ffcda426ae0) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is", 0x7ffcda426940) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test/dir", 0x7ffcda4267a0) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test", 0x7ffcda426610) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long", 0x7ffcda426480) = -1 ENOENT (No such file or directory)
lstat("/home/glen/tmp/php", 0x7ffcda4262f0) = -1 ENOENT (No such file or directory)
lstat("/home/glen/tmp", {st_mode=S_IFDIR|0700, st_size=4096, ...}) = 0
lstat("/home/glen", {st_mode=S_IFDIR|0775, st_size=12288, ...}) = 0
lstat("/home", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
open("/home/glen/tmp/php/long/test/dir/is/here/0", O_WRONLY|O_CREAT|O_TRUNC, 0666) = -1
ENOENT (No such file or directory)
write(2, "PHP Warning: fopen(/home/glen/t"..., 145PHP Warning:
fopen(/home/glen/tmp/php/long/test/dir/is/here/0): failed to open stream: No such file or directory
in Command line code on line 1
lstat("/home/glen/tmp/php/long/test/dir/is/here/1", 0x7ffcda426c80) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is/here", 0x7ffcda426ae0) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is", 0x7ffcda426940) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test/dir", 0x7ffcda4267a0) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test", 0x7ffcda426610) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long", 0x7ffcda426480) = -1 ENOENT (No such file or directory)
lstat("/home/glen/tmp/php", 0x7ffcda4262f0) = -1 ENOENT (No such file or directory)
open("/home/glen/tmp/php/long/test/dir/is/here/1", O_WRONLY|O_CREAT|O_TRUNC, 0666) = -1
ENOENT (No such file or directory)
write(2, "PHP Warning: fopen(/home/glen/t"..., 145PHP Warning:
fopen(/home/glen/tmp/php/long/test/dir/is/here/1): failed to open stream: No such file or directory
in Command line code on line 1
lstat("/home/glen/tmp/php/long/test/dir/is/here/2", 0x7ffcda426c80) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is/here", 0x7ffcda426ae0) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is", 0x7ffcda426940) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test/dir", 0x7ffcda4267a0) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test", 0x7ffcda426610) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long", 0x7ffcda426480) = -1 ENOENT (No such file or directory)
lstat("/home/glen/tmp/php", 0x7ffcda4262f0) = -1 ENOENT (No such file or directory)
open("/home/glen/tmp/php/long/test/dir/is/here/2", O_WRONLY|O_CREAT|O_TRUNC, 0666) = -1
ENOENT (No such file or directory)
write(2, "PHP Warning: fopen(/home/glen/t"..., 145PHP Warning:
fopen(/home/glen/tmp/php/long/test/dir/is/here/2): failed to open stream: No such file or directory
in Command line code on line 1
lstat("/home/glen/tmp/php/long/test/dir/is/here/3", 0x7ffcda426c80) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is/here", 0x7ffcda426ae0) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is", 0x7ffcda426940) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test/dir", 0x7ffcda4267a0) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test", 0x7ffcda426610) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long", 0x7ffcda426480) = -1 ENOENT (No such file or directory)
lstat("/home/glen/tmp/php", 0x7ffcda4262f0) = -1 ENOENT (No such file or directory)
open("/home/glen/tmp/php/long/test/dir/is/here/3", O_WRONLY|O_CREAT|O_TRUNC, 0666) = -1
ENOENT (No such file or directory)
write(2, "PHP Warning: fopen(/home/glen/t"..., 145PHP Warning:
fopen(/home/glen/tmp/php/long/test/dir/is/here/3): failed to open stream: No such file or directory
in Command line code on line 1
lstat("/home/glen/tmp/php/long/test/dir/is/here/4", 0x7ffcda426c80) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is/here", 0x7ffcda426ae0) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is", 0x7ffcda426940) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test/dir", 0x7ffcda4267a0) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test", 0x7ffcda426610) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long", 0x7ffcda426480) = -1 ENOENT (No such file or directory)
lstat("/home/glen/tmp/php", 0x7ffcda4262f0) = -1 ENOENT (No such file or directory)
open("/home/glen/tmp/php/long/test/dir/is/here/4", O_WRONLY|O_CREAT|O_TRUNC, 0666) = -1
ENOENT (No such file or directory)
write(2, "PHP Warning: fopen(/home/glen/t"..., 145PHP Warning:
fopen(/home/glen/tmp/php/long/test/dir/is/here/4): failed to open stream: No such file or directory
in Command line code on line 1
------------------------------------------------------------------------
The remainder of the comments for this report are too long. To view
the rest of the comments, please view the bug report online at
https://bugs.php.net/bug.php?id=38567
--
Edit this bug report at https://bugs.php.net/bug.php?id=38567&edit=1