Re: Speed/Efficiency Question: Large array into DB

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Sounds to me like you are trying to import data. In which case it would be best to create a text file and import it rather than putting PHP in the middle.

But if you are going us to PHP, I would not do individual INSERTs. The overhead will really slow things down with number of inserts you are doing. Use the multiple value insert syntax and do multiple bulk inserts or just one big one. Although doing one big one will end up being an all or nothing deal. With multiple bulk inserts, you'd be able to write to a log file between inserts so you can track it's progress.

I would step through the array, creating the string for inserting (value1, value2, value3, value4),(value1, value2, value3, value4), (value1, value2, value3, value4),...


On Oct 11, 2005, at 12:28 PM, Alan Lord wrote:

Hi All,

Not being a professional programmer I am looking for some advice about
the best way to do the following:

I have a large multidimensional array (up to 7Mb) It contains a list of
categories in a fairly normal Cat_ID and Parent_ID arrangement. The
array also contains a great deal of superfluous information which I want
to ignore.

I wish to walk through the entire array, select 4 key/value pairs from
each sub-array and write those to my Database (Keys are: Cat_ID,
Parent_ID, CatName, CatLevel). I wish to ignore all other key/ values in
each array.

I'm guessing that I should use array_walk_recursive() and apply my test
for valid keys then write this into my db... But, should I do this for
each iteration, or make a new array with just the information I need
then write that into my DB? And do I use a transaction/commit block and try to do the whole thing in one db-write or again, INSERT on each step
through my array?

Oh yes, the key names in the array correspond to field names in my
database.

Sorry it's a bit long-winded...

Thanks in advance for any suggestions.

Alan

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php




--
Brent Baisley
Systems Architect
Landover Associates, Inc.
Search & Advisory Services for Advanced Technology Environments
p: 212.759.6400/800.759.0577

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php


[Index of Archives]     [PHP Home]     [Apache Users]     [PHP on Windows]     [Kernel Newbies]     [PHP Install]     [PHP Classes]     [Pear]     [Postgresql]     [Postgresql PHP]     [PHP on Windows]     [PHP Database Programming]     [PHP SOAP]

  Powered by Linux