This all leads me to think that maybe Squid has something to do with it.
When looking in the cache-spool the mp3-files always seem to have the
right filesize (this is hard to check for certain).
Check the access.log column which lists how many bytes of reply object
sent to the client on each request.
If these are wildly smaller than the requested file size on Apache
then look closer at the cached object size (squidclient or
cachemgr.cgi can show the list)
The cache file size will be around 1-2KB larger than the MP3 file
itself. Which should not be much
First off, thanks for your thorough response!
I've setup a testing webpage at http://fili.nl/squid-testing/
Again the mp3-file only played for 0:15 seconds in the embedded player,
it should play for 3:22.
The mp3-file is 3237011 bytes on disk, but the logged filesize changes
per request in the squid's access-log.
Strangly enough I can now only replicate the problem in the embedded
player and not in a mediaplayer (totem, vlc).
I'm certain though that this has also occured with "JW player" out of
the equation.
BTW. downloading mp3-files has always worked perfectly fine.
There also is enough diskspace and inodes available on the caching
server.
As a test I've instructed squid to prevent the caching of mp3-files
all together by using this refresh_pattern:
refresh_pattern -i \.mp3 0 0% 0
Won't work. refresh_pattern determines how long an object gets to stay
in cache. _after_ its already been stored there.
Use "cache deny XX" to prevent things entering the cache.
Thanks, I thought i might have that line wrong.
I'll look into creating a solid cache deny query, but currently the
.htaccess already does the trick.
and also a .htaccess on a test-domain with these lines:
<IfModule mod_expires.c>
ExpiresActive On
ExpiresDefault "access plus 2 hours"
ExpiresByType audio/mpeg "access"
ExpiresByType audio/mp3 "access"
</IfModule>
This did the trick in never swapping out, however the problem did not
go away.
Those apache lines tell Squid not to cache the object at all.
It's likely a transfer problem then...
Check the HTTP headers sent by Apache. (www.redbot.org) If there are
no explicit file sizes (Content-Length) header and Squid set with
reply object max limits things may get truncated without notification.
Hold on, here might be the problem. When using redbot on my test-file
(http://fili.nl/squid-testing/wacht.mp3) there doesn't seem to be a
Content-Length header, and it also gives out this warning "A ranged
request returned the full rather than partial content."
These are my current defined maximums in squid.conf:
cache_mem 32 MB
maximum_object_size_in_memory 512 KB
maximum_object_size 4096 KB
memory_pools on
memory_pools_limit 50 MB
It put up the full config file on http://fili.nl/squid-testing/squid.conf
I find it very hard to debug this issue and have no idea where to
look next.
Maybe I'm on the wrong track in thicking that Squid is involved.
Maybe yes, maybe no. It's good to know for certain either way.
I'm inclined to think its the NFS filesystem sometimes not giving
Apache all the data. The same check of apache access.log as I mention
for the squid one above should tel you if its that.
It tested it and you're right that the same changes in filesize occurs
in the apache access.log, so it might be NFS.
However, there are a lot of website's running on this cluster of servers
without any problems.
A faulty NFS server would surely cause a bigger problem then there
currently is?
If it is the NFS server, what would be the logical step to take next?
Thanks for all your help so far.
Regards,
Fili
