Re: CSV file

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi,

I don't say that reading the whole file into memory is the best
option, I just say that your approach goes to the disk twice, if you
have a 100MB file, you are reading 200MB from disk (unless there are
some pages cached in physical memory, but you never can know, in the
best case all the pages will be cached and you will only read 100MB
from disk); and mine, if you have 100MB or more physical memory
(available for the script) will only go once. Of course, if you have
low physical memory, in the worst case you will end up reading 200MB
from disk, because all the pages will go to the pagefile.

So, both approaches have a worst case of reading approximately two
times the file size from disk and a best case of reading it
approximately only once from disk, and since you never know the actual
conditions of the memory, you can't say which approach is better
without additional information.

In the general case, I think that your approach is better for large
files and mine for small files, with the size barrier depending on the
memory set up and current conditions at the time the script runs.

This discussion went in a different direction considering the initial
question of Alain, but this could always be useful for someone.

Jonathan

On Thu, Jun 25, 2009 at 5:37 AM, Richard Heyes<richard@xxxxxxx> wrote:
> Hi,
>
>> Well, you are reading the whole file there (and throwing the data you
>> read not assigning the fgets result to anything), and then to store it
>> in the database you need to read it again, so you read the file twice.
>> It will probably better to store the data you read the first time in
>> an array and then store it in the database, that way you read it only
>> once.
>
> No, it's not. If the file is large then you could end up reading megs
> into memory. If physical memory is low then the pagefile will come
> into play and you'll get a lot of disk accesses. Reading 1 line at a
> time is far more efficient and with larger files will be faster.
>
> --
> Richard Heyes
> HTML5 graphing: RGraph (www.rgraph.net - updated 20th June)
> PHP mail: RMail (www.phpguru.org/rmail)
> PHP datagrid: RGrid (www.phpguru.org/rgrid)
> PHP Template: RTemplate (www.phpguru.org/rtemplate)
> PHP SMTP: http://www.phpguru.org/smtp
>

-- 
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php


[Index of Archives]     [PHP Home]     [Apache Users]     [PHP on Windows]     [Kernel Newbies]     [PHP Install]     [PHP Classes]     [Pear]     [Postgresql]     [Postgresql PHP]     [PHP on Windows]     [PHP Database Programming]     [PHP SOAP]

  Powered by Linux