Thanks All. >> >> I am looking at all your suggestions. The file has 9216 entries with 10 columns, but I only need 2 of them really. > > When you say need, does that mean that you are only inserting those two rows into the database, or on the other side of the if..else ? > No, sorry; that was way unclear. I am inserting 6 values. One is a constant, based on the filename (ruinid), and 5 more that come from columns in the file (well 3 and one that is one column in the file, but I insert separately); but I need all rows. These files will belong to projects, and each project will have more than one of these files. So I need the insert time to be a fast as possible. >> >> The crucial one, is the one that I have to split with explode using the “: “ separator. > > crucial in what way? Is this the side that has the two rows or the remaining 9214 rows? See above. Sorry for the confusion. This is crucial, because it represents the actual data. > >> >> The files are about 1MB, that’s why I reached out, I figured it shouldn’t take that long. > > Correct, it shouldn't. Sounds like something (more then likely the DB inserts) that are taking so long. Depending on how your table is built and how much data you already have in there, re-indexing the table could be what is slowing you down. > >> >> Thanks again! >> >> T. >> >> On Mar 18, 2014, at 5:55 PM, Christoph Becker <cmbecker69@xxxxxx> wrote: >> >>> Tiago Hori wrote: >>> >>>> I fairly new at this, so please bear with me. :) >>>> >>>> I am building this web app for a project I am working at where I need to >>>> store and process large amounts of data. >>>> >>>> The data comes in comma delimited style. There are a bunch of headers that >>>> I don't need and then the data. These files contain 96 times 96 entries >>> >>> Do you mean 96 records (rows) with 96 columns each, or do you mean 9216 >>> records? What's the size of the file? >>>> and >>>> I need to parse each one of those. Right now I have something working that >>>> takes about 5 minutes to parse the whole file. >>> >>> I doubt that *parsing* the file takes about 5 minutes. >>> >>>> Any tips on how to make this >>>> run more efficiently would be greatly appreciated. >>> >>> Measure first where you're script spends time, before you optimize in >>> the "wrong" place. A profiler such as Xdebug's or xhprof should come in >>> handy. >>> >>> If you don't have a profiler available, I suggest you replace >>> queryMysql($query) with true, and check how long it takes to process the >>> file. If that is much faster than before (and I presume so), you'll >>> have to optimize the insertion of the values into the database. >>> >>> -- >>> Christoph M. Becker >> >> > > > -- > Jim Lucas > > http://www.cmsws.com/ > http://www.cmsws.com/examples/
