Re: Need For SPEED

Sorry I don't seem to get it. 1. What is"the import function provided in PHPList" ? 2. What do you mean by "initialize each data record " ? Ignatius _________________________ ----- Original Message ----- From: "Creigh Shank" <creigh@medianetpartners.com> To: "Ignatius Reilly" <ignatius.reilly@free.fr> Sent: Thursday, August 21, 2003 7:26 PM Subject: Re: [PHP-DB] Need For SPEED > Thanks. We're using the import function provided in PHPList. Looked at a > more direct method, but we need to properly initialize each data record in > the database. > > Sorry about the cross-posting. Not sure which list can help. > > Creigh > > At 07:13 PM 8/21/2003 +0200, you wrote: > >What is your data load method? What is the type of your table(s)? > > > >With LOAD DATA and text files, I load several million rows in about a > >minute. > > > >Also please don't cross-post. Better first find out what the most suitable > >mailing list is. > > > >Ignatius > >_________________________ > >----- Original Message ----- > >From: "Creigh Shank" <creigh@medianetpartners.com> > >To: "MySQL Users" <mysql@lists.mysql.com>; "PHP-db List" > ><php-db@lists.php.net>; "PHPList Users" <phplist-users@tincan.co.uk>; > >"Michiel Dethmers" <michiel@tincan.co.uk> > >Sent: Thursday, August 21, 2003 6:58 PM > >Subject: [PHP-DB] Need For SPEED > > > > > > > Using an Apache/PHP/MySQL/Linux (Redhat 8.0) solution, PHPList, to create > > > an e-mailing list for our 5.6 million book club members. Unfortunately, > > > the import speed for importing records (at record number 150,000 the rate > > > is about 2,000 records per hour). We're running on the following: > > > > > > P4 (1.5 Ghz), 1.2 Gbytes RAM (650 Mbytes RAM Disk using ramfs), IDE drive > > > (72,00 rpm) > > > > > > So far we've moved the MySQL data files (var/lib/mysql), PHP /tmp and > > > upload directories and PHPlist web site files to RAM Disk (still just > > > testing - not yet dealing with data safety issues). With all of this > > > tuning we're still at only 2,000 records per hour for uploading. > > > > > > We need to be at 100,000 records per hour (uploading and sending seem to > > > run at about the same rate - we need to be able to send to all book club > > > members in the same week). Any suggestions? > > > > > > Creigh > > > > > > (We're planning to run the system on a server with dual Opterons, 8 Gbytes > > > RAM and RAID-5 SCSI drives, but I don't think the additional system > > > horsepower will solve our problem.) > > > > > > > > > -- > > > PHP Database Mailing List (http://www.php.net/) > > > To unsubscribe, visit: http://www.php.net/unsub.php > > > > > > > > >