Re: Re: Efficiently parsing a File

Thanks everyone! I will try the temp file with LOAD DATA INFILE! T. Sent from my iPhone > On Mar 18, 2014, at 9:52 PM, Curtis Maurand <curtis@maurand.com> wrote: > >> On 3/18/2014 7:19 PM, Tiago Hori wrote: >> Jim and Christoph, >> >> Thanks! >> >>>> I am looking at all your suggestions. The file has 9216 entries with >>>> 10 columns, but I only need 2 of them really. >>>> >>>> The crucial one, is the one that I have to split with explode using >>>> the “: “ separator. >>>> >>>> The files are about 1MB, that’s why I reached out, I figured it >>>> shouldn’t take that long. >>> I still presume the performance problem is due to the many separate >>> insert statements (reading such files with fgets() isn't the problem). >>> There is an article regarding the speed of insertions in MySQL >>> databases, with several optimization suggestions: >>> >>> <https://dev.mysql.com/doc/refman/5.7/en/insert-speed.htmlŸa)d
ø% >>> ¢w“®4> >>> >>> HTH >> So, I could use Jim suggestion, but maybe not add the whole 9000 entries of time, correct? >> Would it be a good solution to create separate arrays with every 500 rows or just create one big >> array like Jim suggested and then break the insertion into iterations of 100 rows? >> >> Thanks! >> >> T, > write out a temporary flat file the way you need it to look in the database delimited however > you need it delimited. Then use "LOAD DATA INFILE" to populate the table. LOAD DATA > INFILE is very fast and it's very efficient. It'll load a 1MB file in a fraction of a > second. > > delete the flat file when you're done. > > --Curtis > > -- > PHP General Mailing List (http://www.php.net/) > To unsubscribe, visit: http://www.php.net/unsub.php >