Re: Issue with the huge import script

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 




On Aug 15, 2009, at 9:15 PM, Devendra Jadhav <devendra.in@xxxxxxxxx> wrote:

Hi,

I have to import data from one database to another, I have to import around
100000(1Lac) records.
First I need to check if the record is already imported or not and import
only those records which are not imported.

Here is my logic

$already_imported = get_already_imported_records();
format of the $already_imported is $already_imported[someid] = 'imported';

Now i take all records from another db and iterating through it.

if (!key_exists($already_imported[$new_id])){
       import_function($new_id)
}else{
       echo 'allready imported'.$already_imported[$new_id];
}

Now my script is importing same records for more than one time. I am not
able to get through this issue

Is it because of the size of the records or something else...?

Please suggest me some solution which is faster, safe and easy to code :D

Thanks in advance

--
Devendra Jadhav

What are the databases? Both mysql? Or different systems? How do you define whether the record exists?

Are you running into timeout issues? If so, one trick is to write the page to process 100 or so records at a time and then use JavaScript to reload the page to avoid the timeout issues.

Bastien

Sent from my iPod


--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php


[Index of Archives]     [PHP Home]     [Apache Users]     [PHP on Windows]     [Kernel Newbies]     [PHP Install]     [PHP Classes]     [Pear]     [Postgresql]     [Postgresql PHP]     [PHP on Windows]     [PHP Database Programming]     [PHP SOAP]

  Powered by Linux