Re: Parsing a large file
| From: | Jay Paulson | Date: | Fri, 13 Jan 2006 22:47:11 +0000 |
| Subject: | Re: Parsing a large file | ||
| References: | 1 | Groups: | php.general |
| Request: | Send a blank email to php-general+get-228598@lists.php.net to get a copy of this message | ||
> On Fri, January 13, 2006 3:33 pm, Jay Paulson wrote:
>> $buf = "";
>
> Probably better to initialize it to an empty array();...
Yep right.
>> while (!feof($fhandle)) {
>> $buf[] = fgets($fhandle);
>
> ... since you are going to initialize it to an array here anyway.
>
>> if ($i++ % 10 == 0) {
>
> Buffering 10 lines of text in PHP is probably not going to make a
> significant difference...
This is true. It's what I have written to start with. Basically I'm just
trying to make sure that I'm not hogging system memory with a huge file b/c
there are other apps running at the same time that need system resources as
well. That's the main reason why I'm using a buffer to read the file in and
parse it a little at a time. By all means test it out on your hardware and
see what that buffer needs to be.
> You'll have to test on your hardware to confirm, but between:
>
> 1. Low-level disk IDE buffer
> 2. Operating System disk cache buffers
> 3. C code of PHP source disk cache buffers
>
> your PHP 10-line buffer in an array
> is probably more overhead, and much more complicted code to maintain,
> with no significant benefit.