On 03/18/2014 02:29 PM, Tiago Hori wrote:
Thanks All.
I am looking at all your suggestions. The file has 9216 entries with 10 columns, but I only need 2 of them really.
When you say need, does that mean that you are only inserting those two rows
into the database, or on the other side of the if..else ?
The crucial one, is the one that I have to split with explode using the “: “ separator.
crucial in what way? Is this the side that has the two rows or the remaining
9214 rows?
The files are about 1MB, that’s why I reached out, I figured it shouldn’t take that long.
Correct, it shouldn't. Sounds like something (more then likely the DB
inserts) that are taking so long. Depending on how your table is built and
how much data you already have in there, re-indexing the table could be what
is slowing you down.
Thanks again!
T.
On Mar 18, 2014, at 5:55 PM, Christoph Becker <cmbecker69@xxxxxx> wrote:
Tiago Hori wrote:
I fairly new at this, so please bear with me. :)
I am building this web app for a project I am working at where I need to
store and process large amounts of data.
The data comes in comma delimited style. There are a bunch of headers that
I don't need and then the data. These files contain 96 times 96 entries
Do you mean 96 records (rows) with 96 columns each, or do you mean 9216
records? What's the size of the file?
and
I need to parse each one of those. Right now I have something working that
takes about 5 minutes to parse the whole file.
I doubt that *parsing* the file takes about 5 minutes.
Any tips on how to make this
run more efficiently would be greatly appreciated.
Measure first where you're script spends time, before you optimize in
the "wrong" place. A profiler such as Xdebug's or xhprof should come in
handy.
If you don't have a profiler available, I suggest you replace
queryMysql($query) with true, and check how long it takes to process the
file. If that is much faster than before (and I presume so), you'll
have to optimize the insertion of the values into the database.
--
Christoph M. Becker
--
Jim Lucas
http://www.cmsws.com/
http://www.cmsws.com/examples/
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
