Re: handling large files w/readfile

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Sebastian wrote:
> yea. all the files aren't 100MB though.. some are 2mb (even less) while
> some
> files are over 300MB as well.
> so, does this need to be adjusted depending on the filesize?

I believe that at a certain point, your setting there will be worse for
the system if you make it too big instead of too small, regardless of the
filesize to be written/read.

This is because no matter how big your file is, there are buffers all
along the chain of events between http://php.net/fread and the actual
electrical signals re-aligning the magnetic spots on your hard drive disc.

You're defining a buffer for fread.
PHP may have an internal buffer in a layer deeper than that (probably not,
but *could* have one)
The C library may have a buffer in the next layer down.  Again, probably
not, but it *could* have one.
The library inside the OS that the C library is calling maybe has a buffer.
The OS File System that the OS library calls probably has a buffer.
The physical hardware almost for sure has a buffer, probably several.
If you have fancy hardware and/or RAID software, then there are even more
buffers involved.

And, of course, the RAM available to PHP at the time of doing this work
will drastically affect performance -- If your server is loaded, and this
fread with your (possibly over-sized) buffer forces PHP and/or the OS to
swap something in/out to the hard drive, then you are going to KILL your
system.

At every step of the way, the data being buffered in a faster/cheaper
storage medium will have an effect on your performance.

Hopefully, for most uses, that is an overall Good effect.

But nobody on Earth can tell you for sure for sure that it's all optimum
for *YOUR* application's needs on *YOUR* hardware.

So given your OS, and your hard drive and your file system, and your
files, and the usual load on your server (RAM) there is some "ideal"
number for that collection of hardware/software that you use.

The only way to find your ideal number is to trial and error benchmark it.

I'd try numbers that match up with:
  the cache size on the hard drive
  any internal buffers in PHP's fread() functions
  any buffer in your File System and/or OS of choice
  very large numbers
  very small numbers
  a random assortment of "nice" numbers (1Mb 2Mb 4Mb ...)

Now, if your code has to run on LOTS of different hardware, you'd want to
benchmark on as many different sets of hardware as you can to find the
ideal number for *most* hardware.

Or, you could, I suppose, try to write a system that detects certain
specific categories of hardware/software and choose your buffer size based
on that.

Now, having said all that, the difference to the performance for MOST
applications is going to be negligible.  Find a number that works "fast
enough" for your application (with reasonable testing/benchmarking) and
move on with life.

-- 
Like Music?
http://l-i-e.com/artists.htm

-- 
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php


[Index of Archives]     [PHP Home]     [Apache Users]     [PHP on Windows]     [Kernel Newbies]     [PHP Install]     [PHP Classes]     [Pear]     [Postgresql]     [Postgresql PHP]     [PHP on Windows]     [PHP Database Programming]     [PHP SOAP]

  Powered by Linux