Re: Re: Issue with the huge import script

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Sun, 2009-08-16 at 11:05 +0200, Ralph Deffke wrote:
> because I assume always that a requester got some intelligence, so in that
> case there must be a reasonsble reason why he wants to do it in PHP
> 
> im not like u assuming everybody is a thumb
> 
> ralph
> 
> "Ashley Sheridan" <ash@xxxxxxxxxxxxxxxxxxxx> wrote in message
> news:1250413427.2344.51.camel@xxxxxxxxxxxx
> > On Sun, 2009-08-16 at 04:06 +0200, Ralph Deffke wrote:
> > > Hi,
> > >
> > > this sounds huge, and cries for a sql version of the import.
> > > Are both databases the same? MySQL?
> > >
> > > I give u a draft for MySQL
> > > u export the data u have, then u got a textfile with 100000+ sql
> statments
> > >
> > > in the php script u open the file and iterate over it by line (carefull
> it
> > > could be also ";" in case its a Unix created file on a windows platform)
> > >
> > > line == one SQL insert in table bla bla...
> > >
> > > in the loop then just mysq_query with this line
> > >
> > > if the the someid is an unique index the insert will fail, so only those
> > > records are inserted beeing not already in the database.
> > >
> > > but I think as of the amount off records it doesn't sound like a every
> 10
> > > minutes job, if it is a rara job, just do it with phpMyAdmin
> > >
> > > sorry not pulling out the code, but was a long day behind the keyboard,
> need
> > > some sleep
> > >
> > > ralph_deffke@xxxxxxxx
> > >
> > >
> > >
> > >
> > > "Devendra Jadhav" <devendra.in@xxxxxxxxx> wrote in message
> > > news:be4b00cf0908151815r1c7430d2j8a6cb0da1f10ac00@xxxxxxxxxxxxxxxxx
> > > > Hi,
> > > >
> > > > I have to import data from one database to another, I have to import
> > > around
> > > > 100000(1Lac) records.
> > > > First I need to check if the record is already imported or not and
> import
> > > > only those records which are not imported.
> > > >
> > > > Here is my logic
> > > >
> > > > $already_imported = get_already_imported_records();
> > > > format of the $already_imported is $already_imported[someid] =
> 'imported';
> > > >
> > > > Now i take all records from another db and iterating through it.
> > > >
> > > > if (!key_exists($already_imported[$new_id])){
> > > >         import_function($new_id)
> > > > }else{
> > > >         echo 'allready imported'.$already_imported[$new_id];
> > > > }
> > > >
> > > > Now my script is importing same records for more than one time. I am
> not
> > > > able to get through this issue
> > > >
> > > > Is it because of the size of the records or something else...?
> > > >
> > > > Please suggest me some solution which is faster, safe and easy to code
> :D
> > > >
> > > > Thanks in advance
> > > >
> > > > -- 
> > > > Devendra Jadhav
> > > >
> > >
> > >
> > >
> > You cry for a MySQL version and then revert back to PHP?! Why not just
> > keep the whole thing in MySQL? You can use SQL statements to check
> > whether a record exists before attempting to shove it in the database
> > using a WHERE clause in the INSERT statement or by making one field
> > unique and hiding notices about inserts that are attempting to overwrite
> > that.
> >
> > Thanks,
> > Ash
> > http://www.ashleysheridan.co.uk
> >
> 
> 
> 
There are rare occasions on this list where the best answer is not PHP,
and I believe this is one of them.

Thanks,
Ash
http://www.ashleysheridan.co.uk


-- 
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php


[Index of Archives]     [PHP Home]     [Apache Users]     [PHP on Windows]     [Kernel Newbies]     [PHP Install]     [PHP Classes]     [Pear]     [Postgresql]     [Postgresql PHP]     [PHP on Windows]     [PHP Database Programming]     [PHP SOAP]

  Powered by Linux