Edit report at https://bugs.php.net/bug.php?id=38567&edit=1
ID: 38567
User updated by: glen at delfi dot ee
Reported by: glen at delfi dot ee
Summary: file operations make loads of lstat64() calls and
making code ineffective
Status: Open
Type: Feature/Change Request
Package: *General Issues
-Operating System: PLD Linux
+Operating System:
PHP Version: 5.1.5
Block user comment: N
Private report: N
New Comment:
i do not see any improvement, no matter what php version or cache size is.
post your result proving otherwise, maybe there's something hidden in plain sight.
â php70 -i|grep realpath_cache_size
realpath_cache_size => 4096K => 4096K
â php70 -d realpath_cache_size=128000k -i|grep realpath_cache_size
realpath_cache_size => 128000k => 128000k
â php70 -d realpath_cache_size=128000k --version
PHP 7.0.18 (cli) (built: Apr 24 2017 20:13:36) ( NTS )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.0.0, Copyright (c) 1998-2017 Zend Technologies
â strace php70 -d realpath_cache_size=128000k -r 'for ($i=0;$i<5;$i++)
{$fp=fopen("/home/glen/tmp/php/long/test/dir/is/here/$i","w");fclose($fp);}'
2>&1 |grep /home
lstat("/home/glen/.local/npm/bin/php70", 0x7fff98552520) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test/dir/is/here/0", 0x7fff98551160) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is/here", 0x7fff98550fc0) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is", 0x7fff98550e20) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test/dir", 0x7fff98550c80) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test", 0x7fff98550af0) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long", 0x7fff98550960) = -1 ENOENT (No such file or directory)
lstat("/home/glen/tmp/php", 0x7fff985507d0) = -1 ENOENT (No such file or directory)
lstat("/home/glen/tmp", {st_mode=S_IFDIR|0700, st_size=4096, ...}) = 0
lstat("/home/glen", {st_mode=S_IFDIR|0775, st_size=12288, ...}) = 0
lstat("/home", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
open("/home/glen/tmp/php/long/test/dir/is/here/0", O_WRONLY|O_CREAT|O_TRUNC, 0666) = -1
ENOENT (No such file or directory)
write(2, "PHP Warning: fopen(/home/glen/t"..., 145PHP Warning:
fopen(/home/glen/tmp/php/long/test/dir/is/here/0): failed to open stream: No such file or directory
in Command line code on line 1
lstat("/home/glen/tmp/php/long/test/dir/is/here/1", 0x7fff98551160) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is/here", 0x7fff98550fc0) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is", 0x7fff98550e20) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test/dir", 0x7fff98550c80) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test", 0x7fff98550af0) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long", 0x7fff98550960) = -1 ENOENT (No such file or directory)
lstat("/home/glen/tmp/php", 0x7fff985507d0) = -1 ENOENT (No such file or directory)
open("/home/glen/tmp/php/long/test/dir/is/here/1", O_WRONLY|O_CREAT|O_TRUNC, 0666) = -1
ENOENT (No such file or directory)
write(2, "PHP Warning: fopen(/home/glen/t"..., 145PHP Warning:
fopen(/home/glen/tmp/php/long/test/dir/is/here/1): failed to open stream: No such file or directory
in Command line code on line 1
lstat("/home/glen/tmp/php/long/test/dir/is/here/2", 0x7fff98551160) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is/here", 0x7fff98550fc0) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is", 0x7fff98550e20) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test/dir", 0x7fff98550c80) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test", 0x7fff98550af0) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long", 0x7fff98550960) = -1 ENOENT (No such file or directory)
lstat("/home/glen/tmp/php", 0x7fff985507d0) = -1 ENOENT (No such file or directory)
open("/home/glen/tmp/php/long/test/dir/is/here/2", O_WRONLY|O_CREAT|O_TRUNC, 0666) = -1
ENOENT (No such file or directory)
write(2, "PHP Warning: fopen(/home/glen/t"..., 145PHP Warning:
fopen(/home/glen/tmp/php/long/test/dir/is/here/2): failed to open stream: No such file or directory
in Command line code on line 1
lstat("/home/glen/tmp/php/long/test/dir/is/here/3", 0x7fff98551160) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is/here", 0x7fff98550fc0) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is", 0x7fff98550e20) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test/dir", 0x7fff98550c80) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test", 0x7fff98550af0) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long", 0x7fff98550960) = -1 ENOENT (No such file or directory)
lstat("/home/glen/tmp/php", 0x7fff985507d0) = -1 ENOENT (No such file or directory)
open("/home/glen/tmp/php/long/test/dir/is/here/3", O_WRONLY|O_CREAT|O_TRUNC, 0666) = -1
ENOENT (No such file or directory)
write(2, "PHP Warning: fopen(/home/glen/t"..., 145PHP Warning:
fopen(/home/glen/tmp/php/long/test/dir/is/here/3): failed to open stream: No such file or directory
in Command line code on line 1
lstat("/home/glen/tmp/php/long/test/dir/is/here/4", 0x7fff98551160) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is/here", 0x7fff98550fc0) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is", 0x7fff98550e20) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test/dir", 0x7fff98550c80) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test", 0x7fff98550af0) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long", 0x7fff98550960) = -1 ENOENT (No such file or directory)
lstat("/home/glen/tmp/php", 0x7fff985507d0) = -1 ENOENT (No such file or directory)
open("/home/glen/tmp/php/long/test/dir/is/here/4", O_WRONLY|O_CREAT|O_TRUNC, 0666) = -1
ENOENT (No such file or directory)
write(2, "PHP Warning: fopen(/home/glen/t"..., 145PHP Warning:
fopen(/home/glen/tmp/php/long/test/dir/is/here/4): failed to open stream: No such file or directory
in Command line code on line 1
â
Previous Comments:
------------------------------------------------------------------------
[2017-09-21 11:06:34] cmb@php.net
That appears to be related to the ini directive realpath_cache_size[1], which
defaults to only 16k before PHP 7.0.16. Try to increase the setting to 4096k,
for instance.
[1] <http://www.php.net/manual/en/ini.core.php#ini.realpath-cache-size>
------------------------------------------------------------------------
[2017-01-23 09:43:47] glen at delfi dot ee
â php --version
PHP 5.6.27 (cli) (built: Nov 21 2016 23:08:15)
Copyright (c) 1997-2016 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2016 Zend Technologies
with Zend OPcache v7.0.6-dev, Copyright (c) 1999-2016, by Zend Technologies
with Xdebug v2.4.1, Copyright (c) 2002-2016, by Derick Rethans
â php -r 'var_dump(ini_get("open_basedir"));'
Command line code:1:
string(0) ""
â strace php -r 'for ($i=0;$i<5;$i++)
{$fp=fopen("/home/glen/tmp/php/long/test/dir/is/here/$i","w");fclose($fp);}'
2>&1 |grep /home
open("/home/glen/.config/ImageMagick/log.xml", O_RDONLY) = -1 ENOENT (No such file or
directory)
open("/home/glen/.magick/log.xml", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/home/glen/.config/ImageMagick/policy.xml", O_RDONLY) = -1 ENOENT (No such file or
directory)
open("/home/glen/.magick/policy.xml", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/home/glen/.krb5/config", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/home/glen/.snmp", 0x7ffcda428cf0) = -1 ENOENT (No such file or directory)
stat("/home/glen/.snmp", 0x7ffcda428cf0) = -1 ENOENT (No such file or directory)
stat("/home/glen/.snmp/tls/ca-certs", 0x7ffcda426870) = -1 ENOENT (No such file or
directory)
stat("/home/glen/.snmp/tls/certs", 0x7ffcda426870) = -1 ENOENT (No such file or directory)
stat("/home/glen/.snmp/tls/private", 0x7ffcda426870) = -1 ENOENT (No such file or
directory)
open("/home/glen/.snmp/mibs", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = -1 ENOENT (No
such file or directory)
stat("/home/glen/.snmp", 0x7ffcda428d00) = -1 ENOENT (No such file or directory)
stat("/home/glen/.snmp", 0x7ffcda428d00) = -1 ENOENT (No such file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is/here/0", 0x7ffcda426c80) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is/here", 0x7ffcda426ae0) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is", 0x7ffcda426940) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test/dir", 0x7ffcda4267a0) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test", 0x7ffcda426610) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long", 0x7ffcda426480) = -1 ENOENT (No such file or directory)
lstat("/home/glen/tmp/php", 0x7ffcda4262f0) = -1 ENOENT (No such file or directory)
lstat("/home/glen/tmp", {st_mode=S_IFDIR|0700, st_size=4096, ...}) = 0
lstat("/home/glen", {st_mode=S_IFDIR|0775, st_size=12288, ...}) = 0
lstat("/home", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
open("/home/glen/tmp/php/long/test/dir/is/here/0", O_WRONLY|O_CREAT|O_TRUNC, 0666) = -1
ENOENT (No such file or directory)
write(2, "PHP Warning: fopen(/home/glen/t"..., 145PHP Warning:
fopen(/home/glen/tmp/php/long/test/dir/is/here/0): failed to open stream: No such file or directory
in Command line code on line 1
lstat("/home/glen/tmp/php/long/test/dir/is/here/1", 0x7ffcda426c80) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is/here", 0x7ffcda426ae0) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is", 0x7ffcda426940) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test/dir", 0x7ffcda4267a0) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test", 0x7ffcda426610) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long", 0x7ffcda426480) = -1 ENOENT (No such file or directory)
lstat("/home/glen/tmp/php", 0x7ffcda4262f0) = -1 ENOENT (No such file or directory)
open("/home/glen/tmp/php/long/test/dir/is/here/1", O_WRONLY|O_CREAT|O_TRUNC, 0666) = -1
ENOENT (No such file or directory)
write(2, "PHP Warning: fopen(/home/glen/t"..., 145PHP Warning:
fopen(/home/glen/tmp/php/long/test/dir/is/here/1): failed to open stream: No such file or directory
in Command line code on line 1
lstat("/home/glen/tmp/php/long/test/dir/is/here/2", 0x7ffcda426c80) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is/here", 0x7ffcda426ae0) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is", 0x7ffcda426940) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test/dir", 0x7ffcda4267a0) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test", 0x7ffcda426610) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long", 0x7ffcda426480) = -1 ENOENT (No such file or directory)
lstat("/home/glen/tmp/php", 0x7ffcda4262f0) = -1 ENOENT (No such file or directory)
open("/home/glen/tmp/php/long/test/dir/is/here/2", O_WRONLY|O_CREAT|O_TRUNC, 0666) = -1
ENOENT (No such file or directory)
write(2, "PHP Warning: fopen(/home/glen/t"..., 145PHP Warning:
fopen(/home/glen/tmp/php/long/test/dir/is/here/2): failed to open stream: No such file or directory
in Command line code on line 1
lstat("/home/glen/tmp/php/long/test/dir/is/here/3", 0x7ffcda426c80) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is/here", 0x7ffcda426ae0) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is", 0x7ffcda426940) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test/dir", 0x7ffcda4267a0) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test", 0x7ffcda426610) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long", 0x7ffcda426480) = -1 ENOENT (No such file or directory)
lstat("/home/glen/tmp/php", 0x7ffcda4262f0) = -1 ENOENT (No such file or directory)
open("/home/glen/tmp/php/long/test/dir/is/here/3", O_WRONLY|O_CREAT|O_TRUNC, 0666) = -1
ENOENT (No such file or directory)
write(2, "PHP Warning: fopen(/home/glen/t"..., 145PHP Warning:
fopen(/home/glen/tmp/php/long/test/dir/is/here/3): failed to open stream: No such file or directory
in Command line code on line 1
lstat("/home/glen/tmp/php/long/test/dir/is/here/4", 0x7ffcda426c80) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is/here", 0x7ffcda426ae0) = -1 ENOENT (No such
file or directory)
lstat("/home/glen/tmp/php/long/test/dir/is", 0x7ffcda426940) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test/dir", 0x7ffcda4267a0) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long/test", 0x7ffcda426610) = -1 ENOENT (No such file or
directory)
lstat("/home/glen/tmp/php/long", 0x7ffcda426480) = -1 ENOENT (No such file or directory)
lstat("/home/glen/tmp/php", 0x7ffcda4262f0) = -1 ENOENT (No such file or directory)
open("/home/glen/tmp/php/long/test/dir/is/here/4", O_WRONLY|O_CREAT|O_TRUNC, 0666) = -1
ENOENT (No such file or directory)
write(2, "PHP Warning: fopen(/home/glen/t"..., 145PHP Warning:
fopen(/home/glen/tmp/php/long/test/dir/is/here/4): failed to open stream: No such file or directory
in Command line code on line 1
------------------------------------------------------------------------
[2017-01-23 09:42:05] glen at delfi dot ee
â php -r 'var_dump(ini_get("open_basedir"));'
Command line code:1:
string(0) ""
------------------------------------------------------------------------
[2017-01-21 08:48:04] requinix@php.net
Note this is subject to open_basedir: if enabled then PHP has to stat everything every time (in case
it changed) but if not then PHP can reuse previous results.
------------------------------------------------------------------------
[2017-01-21 08:36:50] requinix@php.net
Doesn't seem to be an issue using PHP 7.0.13 on Ubuntu. With /a/b/c/d/$i it stats /a/b/c/d/0
down to /a, then /a/b/c/d/1, 2, etc.
------------------------------------------------------------------------
The remainder of the comments for this report are too long. To view
the rest of the comments, please view the bug report online at
https://bugs.php.net/bug.php?id=38567
--
Edit this bug report at https://bugs.php.net/bug.php?id=38567&edit=1