Re: Efficiently parsing a File
| From: | Tiago Hori | Date: | Tue, 18 Mar 2014 23:19:06 +0000 |
| Subject: | Re: Efficiently parsing a File | ||
| References: | 1 2 3 4 | Groups: | php.general |
| Request: | Send a blank email to php-general+get-323080@lists.php.net to get a copy of this message | ||
Jim and Christoph,
Thanks!
>> I am looking at all your suggestions. The file has 9216 entries with
>> 10 columns, but I only need 2 of them really.
>>
>> The crucial one, is the one that I have to split with explode using
>> the “: “ separator.
>>
>> The files are about 1MB, that’s why I reached out, I figured it
>> shouldn’t take that long.
>
> I still presume the performance problem is due to the many separate
> insert statements (reading such files with fgets() isn't the problem).
> There is an article regarding the speed of insertions in MySQL
> databases, with several optimization suggestions:
>
>
> <https://dev.mysql.com/doc/refman/5.7/en/insert-speed.html>
>
> HTH
>
So, I could use Jim suggestion, but maybe not add the whole 9000 entries of time, correct? Would it
be a good solution to create separate arrays with every 500 rows or just create one big array like
Jim suggested and then break the insertion into iterations of 100 rows?
Thanks!
T,
> --
> Christoph M. Becker
>
>