Re: Speed/Efficiency Question: Large array into DB

From: Date: Tue, 11 Oct 2005 18:35:24 +0000
Subject: Re: Speed/Efficiency Question: Large array into DB
References: 1  Groups: php.general 
Request: Send a blank email to php-general+get-223921@lists.php.net to get a copy of this message
Sounds to me like you are trying to import data. In which case it would be best to create a text file and import it rather than putting PHP in the middle. But if you are going us to PHP, I would not do individual INSERTs. The overhead will really slow things down with number of inserts you are doing. Use the multiple value insert syntax and do multiple bulk inserts or just one big one. Although doing one big one will end up being an all or nothing deal. With multiple bulk inserts, you'd be able to write to a log file between inserts so you can track it's progress. I would step through the array, creating the string for inserting (value1, value2, value3, value4),(value1, value2, value3, value4),(value1, value2, value3, value4),... On Oct 11, 2005, at 12:28 PM, Alan Lord wrote:
Hi All, Not being a professional programmer I am looking for some advice about the best way to do the following: I have a large multidimensional array (up to 7Mb) It contains a list of categories in a fairly normal Cat_ID and Parent_ID arrangement. The array also contains a great deal of superfluous information which I want to ignore. I wish to walk through the entire array, select 4 key/value pairs from each sub-array and write those to my Database (Keys are: Cat_ID, Parent_ID, CatName, CatLevel). I wish to ignore all other key/values in each array. I'm guessing that I should use array_walk_recursive() and apply my test for valid keys then write this into my db... But, should I do this for each iteration, or make a new array with just the information I need then write that into my DB? And do I use a transaction/commit block and try to do the whole thing in one db-write or again, INSERT on each step through my array? Oh yes, the key names in the array correspond to field names in my database. Sorry it's a bit long-winded... Thanks in advance for any suggestions. Alan --PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
-- Brent Baisley Systems Architect Landover Associates, Inc. Search & Advisory Services for Advanced Technology Environments p: 212.759.6400/800.759.0577

« previous php.general (#223921) next »