On Sat, October 22, 2005 5:04 pm, tg-php@xxxxxxxxxxxxxxxxxxxxxx wrote: > Does it make a difference if instead of one record at a time, you pull > 10, 20, 100... might speed up your process a little and if it > doesn't use any more memory, then why not? I could make a buffer of N lines and read them, I guess. I kinda figured at 1 line a second, PHP and the OS would have no problem buffering from the disk. > Another thought.. can you use database replication here? So instead > of dumping all the records and processing and inserting the ones that > are new, you'd only be acting on what's changed since the last backup. > With real replication, you can do this realtime (or at least > semi-realtime), but you could simulate replication in your PHP script > as well. This data's coming from an external source. Replication is probably not gonna happen. Though I am going to ask for a timestamp column so I can ignore rows that haven't changed since my last import. That should kill about 90% of the MySQL processing, which seems to be the biggest CPU sink here. If that happens, I can probably change the sleep to usleep and a quarter second or so. -- Like Music? http://l-i-e.com/artists.htm -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
