Re: Need performance help on importer using DB_DataObject

a) Make sure you have the latest release - there was some memory fixes in that. b) do 3 imports and make sure that print_r($GLOBALS['_DB_DATAOBJECT']); is not getting to big.. use clone (or it's emulator in dataobjects:) #time php -r 'require_once "DB/DataObject.php"; $x = new DB_DataObject; for($i=0;$i<15000;$i++) { $xx = clone($x); $xx->a =4; }'

real    0m1.044s
user    0m0.650s
sys     0m0.190s

#time php -r 'require_once "DB/DataObject.php"; $x = new DB_DataObject; for($i=0;$i<15000;$i++) { $xx = new DB_DataObject; $xx->a =4; }'

real    0m2.690s
user    0m2.130s
sys     0m0.190s

The new is really expensive :) - cloning is cheap... if all that fails - see how expensive Dataobjects really is by timing the $member->query("INSERT INTO member (first_name) VALUES ('test')"); Let me know how you get on. Regards Alan Mike Zornek wrote:

So I've been using DB_DataObject in myproject and am now working on importing stuff from our old db. I have an ~ 13,000 line CSV file which I am iterating through. I will first unset the var I use to hold the object, do some manipulation and then create, set and insert into the db. While() { unset($member); ... $member = new DataObjects_Member(); $member->setMember_id($id); $member->setMember_first_name($first_name); $member->setMember_last_name($last_name); $member->setMember_suffix($suffix); $member->setMember_birth_date($birthday); $member->setMember_block_fap($blockfap); $member->setMember_description($description); $member->setMember_comments($comments); $member->setMember_company($company); $member->setMember_last_modified($last_modified); $member->setMember_primary_address_id($address1_id); $member->insert(); } My first concern is how much memory/cpu this uses. True the importer is never used in production but I'll have to use it a lot over the next few weeks and as of right now it takes a little over 17 minutes to do it's thing. I'm on a dual 1.8 GHz G5 Powermac. Is there anyway to properly destroy the object? Or maybe reset it, so that I may use only one instant to insert all my rows--thus cutting down on memory usage? I run the importer from the command line and notice that the php process uses about 90% of the CPU while importing. The mysqld process jumps around 3-5%. Does this sound right? Should I do anything to Mysql to help speed up the import? Turn off indexing perhaps? What do you think is the bottle neck here? Thanks! ~ Mike ----- Mike Zornek Web Designer, Media Developer, Programmer and Geek Personal site: <http://MikeZornek.com>

-- Can you help out? Need Consulting Services or Know of a Job? http://www.akbkhome.com