>>>>> "Creigh" == Creigh Shank <creigh@medianetpartners.com> writes: Creigh> Using an Apache/PHP/MySQL/Linux (Redhat 8.0) solution, Creigh> PHPList, to create an e-mailing list for our 5.6 million book Creigh> club members. Unfortunately, the import speed for importing Creigh> records (at record number 150,000 the rate is about 2,000 Creigh> records per hour). We're running on the following: Creigh> P4 (1.5 Ghz), 1.2 Gbytes RAM (650 Mbytes RAM Disk using Creigh> ramfs), IDE drive (72,00 rpm) Creigh> So far we've moved the MySQL data files (var/lib/mysql), PHP Creigh> /tmp and upload directories and PHPlist web site files to RAM Creigh> Disk (still just testing - not yet dealing with data safety Creigh> issues). With all of this tuning we're still at only 2,000 Creigh> records per hour for uploading. Creigh> We need to be at 100,000 records per hour (uploading and Creigh> sending seem to run at about the same rate - we need to be Creigh> able to send to all book club members in the same week). Any Creigh> suggestions? What is your data source for the records? Are they in a bunch of flat files? If so, cut out the overhead of Apache/PHP and use Perl with the DBI modules to insert the records directly into your database. Also, make sure you have the proper indexes for your database that can be a killer. Also, for that size of DB, I'd make sure you're using InnoDB tables. What are your mysql settings like as well? You can tune them up quite a bit since your import will be the big problem at first, but then it will mostly be just reads on the tables when you send out emails. John John Stoffel - Senior Unix Systems Administrator - Lucent Technologies stoffel@lucent.com - http://www.lucent.com - 978-952-7830 -- PHP Database Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
