#19971 [Com]: file() function extremally slow
| From: | jamie at jamiebecker dot com | Date: | Sat, 25 Jan 2003 01:10:49 +0000 |
| Subject: | #19971 [Com]: file() function extremally slow | ||
| References: | 1 | Groups: | php.bugs |
| Request: | Send a blank email to php-bugs+get-31196@lists.php.net to get a copy of this message | ||
ID: 19971
Comment by: jamie@jamiebecker.com
Reported By: sazonenkov@yandex.ru
Status: Closed
Bug Type: *Directory/Filesystem functions
Operating System: any
PHP Version: 4.3.0-dev
New Comment:
Please reopen.
PHP 4.2.2 and 4.2.1 appeared to cause delays as well but
PHP 4.3.0 made Squirrelmail unusable. phpMyAdmin appeared
to be affected as well. Downgrading to PHP 4.0.6 resolved
problem. Not clear if this file() function is culprit, but
system load remained low and Cyrus and MySQL do not appear
to be culprit of the application-level slowdown. (Command
line clients do not experience problems.) NOTE: This does
NOT appear to be resolved by CVS snapshot as of 2002 01-23,
which is contrary to above PHP bug!
Additional information:
Gentoo Linux 1.2, nightly update (gcc 2.95)
Cyrus IMAPd 2.1.11
MySQL 3.23.54a
PHP 4.3.0, 4.2.2, 4.2.1
SquirrelMail 1.2.10
phpMyAdmin 2.3.2 (Gentoo rev 1)
Please also see
http://bugs.gentoo.org/show_bug.cgi?id=14513
Previous Comments:
------------------------------------------------------------------------
[2002-10-18 15:40:14] iliaa@php.net
This bug has been fixed in CVS.
In case this was a PHP problem, snapshots of the sources are packaged
every three hours; this change will be in the next snapshot. You can
grab the snapshot at http://snaps.php.net/.
In case this was a documentation problem, the fix will show up soon at
http://www.php.net/manual/.
In case this was a PHP.net website problem, the change will show
up on the PHP.net site and on the mirror sites in short time.
Thank you for the report, and for helping us make PHP better.
------------------------------------------------------------------------
[2002-10-18 09:59:58] iliaa@php.net
I got the following times from a tests I've run on PHP.
The test involved openning a 2.5 meg binary file 1000 times.
The results are everages of several runs.
file_get_contents() - 30.4 seconds (4.3 HEAD)
file() - 99.2 seconds (4.3 HEAD)
file() - 40.3 seconds (4.2.3)
Looks like a VERY serious performance loss to me, I'd even go as far
as to say this is a critical issue that should be resolved before
release.
For reference perpouses a simple C program that freads() entire file to
memory took only 18.5 seconds to run (1000 runs on the same file).
------------------------------------------------------------------------
[2002-10-18 07:04:36] yohgaki@php.net
fgets() is fast.
yohgaki@dev DEV]$ cat t.php
<?php
$filename = '/var/log/httpd/error_log.1';
require("Benchmark/Timer.php");
$time = new Benchmark_Timer;
$time -> setMarker('Start');
$fp = fopen($filename,'r');
for ($i = 0; $i<100; $i++)
$s = explode("\n", fread($fp, filesize($filename)));
fclose($fp);
$time -> setMarker('fread');
for ($i = 0; $i<100; $i++)
$s = file($filename);
$time -> setMarker('file');
for ($i = 0; $i<100; $i++)
$s = file_get_contents($filename);
$time -> setMarker('file_get_contents');
$fp = fopen($filename,'r');
for ($i = 0; $i<100; $i++)
$s = fgets($fp,filesize($filename));
$time -> setMarker('fgets');
fclose($fp);
[yohgaki@dev DEV]$ ./sapi/cli/php t.php
-------------------------------------------------------------------------
marker time index ex time
perct
-------------------------------------------------------------------------
Start 1034942600.41311100 -
0.00%
-------------------------------------------------------------------------
fread 1034942600.43045000 0.017338991165161
2.68%
-------------------------------------------------------------------------
file 1034942600.95939400 0.52894401550293
81.66%
-------------------------------------------------------------------------
file_get_contents 1034942601.04506900 0.085675001144409
13.23%
-------------------------------------------------------------------------
fgets 1034942601.06049200 0.015423059463501
2.38%
-------------------------------------------------------------------------
Stop 1034942601.06084500 0.00035297870635986
0.05%
-------------------------------------------------------------------------
total - 0.64773404598236
100.00%
-------------------------------------------------------------------------
[yohgaki@dev DEV]$
------------------------------------------------------------------------
[2002-10-18 06:50:52] sazonenkov@yandex.ru
Damn Mozilla wont work with buffer normally :(
Lets try again.
Problem: read from a text file into array
1. using fread()+explode()
2. using file()
Here my tests
<?php
require("Benchmark/Timer.php");
$time = new Benchmark_Timer;
$time -> setMarker('Start');
$fp = fopen("access.log", "r");
$s = explode("\n", fread($fp, filesize("access.log") ) ) ;
fclose($fp);
$time -> setMarker('fread+explode');
$s = file("access.log");
$time -> setMarker('file');
$time -> setMarker('Stop');
$time -> display();
?>
And results
---------------------------------------------------
time index ex time %
Start 1034941339.44905500 - 0.00%
fread+explode 1034941340.05736200 0.608307 34.72%
file 1034941341.20114900 1.143787 65.28%
Stop 1034941341.20128500 0.000136 0.01%
total - 1.752230 100.00%
---------------------------------------------------
Is it right?
I dont think so.
------------------------------------------------------------------------
[2002-10-18 06:25:17] yohgaki@php.net
file_get_contents() is alow slow. It's better than file(), though.
[yohgaki@dev DEV]$ cat t.php
<?php
$filename = '/var/log/httpd/error_log.1';
require("Benchmark/Timer.php");
$time = new Benchmark_Timer;
$time -> setMarker('Start');
$fp = fopen($filename,'r');
for ($i = 0; $i<100; $i++)
$s = fread($fp, filesize($filename));
fclose($fp);
$time -> setMarker('fread');
for ($i = 0; $i<100; $i++)
$s = file($filename);
$time -> setMarker('file');
for ($i = 0; $i<100; $i++)
$s = file_get_contents($filename);
$time -> setMarker('file_get_contents');
$time -> setMarker('Stop');
$time -> display();
?>
[yohgaki@dev DEV]$ ./sapi/cli/php t.php
-------------------------------------------------------------------------
marker time index ex time
perct
-------------------------------------------------------------------------
Start 1034940250.73024800 -
0.00%
-------------------------------------------------------------------------
fread 1034940250.74181700 0.011569023132324
1.86%
-------------------------------------------------------------------------
file 1034940251.26557900 0.52376198768616
84.25%
-------------------------------------------------------------------------
file_get_contents 1034940251.35169600 0.086117029190063
13.85%
-------------------------------------------------------------------------
Stop 1034940251.35192900 0.00023293495178223
0.04%
-------------------------------------------------------------------------
total - 0.62168097496033
100.00%
-------------------------------------------------------------------------
------------------------------------------------------------------------
The remainder of the comments for this report are too long. To view
the rest of the comments, please view the bug report online at
http://bugs.php.net/19971
--
Edit this bug report at http://bugs.php.net/?id=19971&edit=1