Need performance help on importer using DB_DataObject
| From: | Mike Zornek | Date: | Tue, 08 Jun 2004 15:50:42 +0000 |
| Subject: | Need performance help on importer using DB_DataObject | ||
| Groups: | php.pear.general | ||
| Request: | Send a blank email to pear-general+get-13012@lists.php.net to get a copy of this message | ||
So I've been using DB_DataObject in myproject and am now working on
importing stuff from our old db. I have an ~ 13,000 line CSV file which I am
iterating through. I will first unset the var I use to hold the object, do
some manipulation and then create, set and insert into the db.
While() {
unset($member);
...
$member = new DataObjects_Member();
$member->setMember_id($id);
$member->setMember_first_name($first_name);
$member->setMember_last_name($last_name);
$member->setMember_suffix($suffix);
$member->setMember_birth_date($birthday);
$member->setMember_block_fap($blockfap);
$member->setMember_description($description);
$member->setMember_comments($comments);
$member->setMember_company($company);
$member->setMember_last_modified($last_modified);
$member->setMember_primary_address_id($address1_id);
$member->insert();
}
My first concern is how much memory/cpu this uses. True the importer is
never used in production but I'll have to use it a lot over the next few
weeks and as of right now it takes a little over 17 minutes to do it's
thing. I'm on a dual 1.8 GHz G5 Powermac.
Is there anyway to properly destroy the object? Or maybe reset it, so that I
may use only one instant to insert all my rows--thus cutting down on memory
usage?
I run the importer from the command line and notice that the php process
uses about 90% of the CPU while importing. The mysqld process jumps around
3-5%. Does this sound right?
Should I do anything to Mysql to help speed up the import? Turn off indexing
perhaps?
What do you think is the bottle neck here? Thanks!
~ Mike
-----
Mike Zornek
Web Designer, Media Developer, Programmer and Geek
Personal site: <http://MikeZornek.com>