#22744 [Opn]: Split hangs on something that 'file' does quickly
| From: | pollita@php.net | Date: | Tue, 01 Apr 2003 06:40:46 +0000 |
| Subject: | #22744 [Opn]: Split hangs on something that 'file' does quickly | ||
| References: | 1 | Groups: | php.bugs |
| Request: | Send a blank email to php-bugs+get-36892@lists.php.net to get a copy of this message | ||
ID: 22744
Updated by: pollita@php.net
Reported By: tim at timcrider dot com
Status: Open
Bug Type: Performance problem
Operating System: Red Hat 8.0
PHP Version: 4.3.2RC1
New Comment:
3MB is not a trivial amount of data to do an operation like split() on.
It's also worth noting (per your summary) that file() does not read
all the data into a single var and perform a split, that sort of
wasteful processing is left to scripts.
The first response I'd give (which addresses what you're asking for)
would be to say that you should be using explode(), not split(). Using
regex for this task is like using a hammer to close a CD tray.
But that's not the best way to do this anyway.
You notice how count($lstk) == 30626 ? Have you looked at the contents
of the $lstk array?
I think print_r($lstk) might surprise you. Hint: Wez already told you
what you'd find.
Previous Comments:
------------------------------------------------------------------------
[2003-03-18 15:27:24] tim at timcrider dot com
Here is a quick size test I did:
---[ size.php ]---
#!/bin/php
<?php
$mybin = "/usr/local/mysql/bin/mysqlbinlog";
$mylogpath = "/var/lib/mysql/";
$dolog = $mylogpath."towely-bin.010";
$sys =
mysqlbinlog $dolog;
print "Backtick :: ".strlen($sys)."\n";
ob_start();
system("mysqlbinlog $dolog");
$sys = ob_get_contents();
ob_end_clean();
print "System :: ".strlen($sys)."\n";
$sys = exec("mysqlbinlog $dolog", $lstk);
print "Exec :: ".strlen($sys)."\n";
print count($lstk)."\n";
?>
---[ End size.php ]---
Here's the output
---[ Begin Output ]---
(towely) phpbug# ./sizes.php
Backtick :: 3120815
System :: 3120815
Exec :: 79
30626
--[ End Output ]---
If there code in particular you'd like me to run, you could email it to
me @ tim at timcrider dot com and I'll post the output of those tests
here.
------------------------------------------------------------------------
[2003-03-18 15:07:43] wez@php.net
http://www.php.net/manual/en/function.exec.php
The second parameter will be filled with each line of the command.
I'm still interested to hear what the size of the data is:
echo strlen($sys);
------------------------------------------------------------------------
[2003-03-18 13:03:40] tim at timcrider dot com
I apologize again. This is what 'works.php' SHOULD look like:
---[ works.php ]---
#!/bin/php
<?php
$mybin = "/usr/local/mysql/bin/mysqlbinlog";
$mylogpath = "/var/lib/mysql/";
$dolog = $mylogpath."towely-bin.010";
$tmpfname = tempnam ("/tmp", "mybinlogger");
$fp = fopen($tmpfname, "w");
fwrite($fp, $mybin $dolog);
fclose($fp);
$lstk = file($tmpfname);
unset($tmpfname);
print count($lstk)."\n";
?>
--[ End works.php ]---
------------------------------------------------------------------------
[2003-03-18 13:02:00] tim at timcrider dot com
I apologize the second script is bogus. Here is the fix:
---[ works.php ]---
#!/bin/php
<?php
$mybin = "/usr/local/mysql/bin/mysqlbinlog";
$mylogpath = "/var/lib/mysql/";
$dolog = $mylogpath."towely-bin.010";
$lstk = exec("mysqlbinlog $dolog");
print_r($lstk);
print count($lstk)."\n";
?>
(towely) phpbug# more test.php
#!/bin/php
<?php
$mybin = "/usr/local/mysql/bin/mysqlbinlog";
$mylogpath = "/var/lib/mysql/";
$dolog = $mylogpath."towely-bin.010";
$tmpfname = tempnam ("/tmp", "mybinlogger");
$fp = fopen($tmpfname, "w");
fwrite($fp, $mybin $dolog);
fclose($fp);
$lstk = file($tmpfname);
unset($tmpfname);
print count($lstk)."\n";
?>
--[ End works.php ]---
---[ broken.php ]---
#!/bin/php
<?php
$mybin = "/usr/local/mysql/bin/mysqlbinlog";
$mylogpath = "/var/lib/mysql/";
$dolog = $mylogpath."towely-bin.010";
$sys = mysqlbinlog $dolog;
$lstk = split("\n", $sys);
print_r($lstk);
print count($lstk)."\n";
?>
---[ End broken.php ]---
Using exec does not give me the output that I need to do this. exec
only returns the last line of the output. I'm trying to get all of the
output, which has line breaks at the end of every row.
I did a dump of this server using 'php -i > php_ini.txt
and can be seen here.
http://timcrider.com/php_ini.txt
If there is anything else you need me to do let me know.
Tim
------------------------------------------------------------------------
[2003-03-18 12:41:51] wez@php.net
Your second test script look bogus to me.
Have you considered using http://www.php.net/manual/en/ref.exec.php,
which gives the lines as an array anyway?
How big is the logfile? Do you have magic_quotes enabled?
If you leave it running, how long does it take to complete and how much
memory does it use?
------------------------------------------------------------------------
The remainder of the comments for this report are too long. To view
the rest of the comments, please view the bug report online at
http://bugs.php.net/22744
--
Edit this bug report at http://bugs.php.net/?id=22744&edit=1