On Mon, 27 Jun 2005, Brett Glass wrote:
It is unclear to this author why the "range_offset_limit" parameter is defined in the way it is, since I can think of no case in which it's desirable to fetch the entire file up to and including a range if the results are overwhelmingly likely to be discarded immediately thereafter.
range_offset_limit was originally designed in response to Acrobat Reader fetching the PDF TOC at the end of the file and then the rest.
It is disabled by default as it is known to cause bandwidth waste in some situations.
(Perhaps it's meant to deal with servers that don't implement range requests.) It's also unclear why the parameters whose names begin "quick_abort" are applied to such fetches at all, since the whole point of prefetching the file is, well, to prefetch it even after the client has gotten the range of bytes it wants.
Many things are unclear with how the fetch of the object while there is no clients listening should work.
And I agree that the quick_abort feature is both strange and not always optimal.
Unfortunately, this causes yet another problem to surface. If the file in question is larger than "maximum_object_size" (the maximum object size defined for the cache), Squid retrieves the entire file in response to each request but then fails to store it!
Is this verified with 2.5.STABLE10, or only earlier versions?
1. Modify Squid so that before it attempts to prefetch an entire object in response to a range request, it first checks the size of the object against "maximum_object_size". It should also perform all the other checks it normally performs to determine whether an object is cacheable (including evaluation of the ACLs, if any, used with the "no_cache" parameter). If the entire file is not cacheable for any reason whatsoever, the range request should be passed through verbatim. (This seems like common sense, but it is apparently not done now.)
Problem with this is that to do this Squid has to forward the request to the web server, at which point it is too late to change the request to/from being a partial request.
2. Allow the "maximum_object_size" parameter to be selected, for each transfer, via an ACL. (Fortuitously, the standard syntax of the Squid configuration file both allows this and provides backward compatibility in the case where no ACL is specified. See the "tcp_outgoing_address" parameter, which acquired the ability to use ACLs only recently and is backward compatible with configuration files that don't exploit this new capability.) With this modification, an administrator could allow the caching of large Windows Update files but not large files from other sources.
Problem with this is developer time. There currently is very few developers working on Squid.
3. If a range transfer is to be expanded into a transfer of the entire file, exempt the transfer from the "quick_abort" mechanism. (After all, it's normal for the client to disconnect after it receives the data it has requested.)
Same as 2.
4. Encourage Microsoft to modify Windows Update so that it can "discover" a server on which updates are preloaded or cached. Currently, SUS and WUS require modification to a client machine's registry; this is practical for organizations with IT staffs but not for ISPs.
Historically Windows Update has became increasingly less and less friendly to caching. As a result it's my impression they do not want Windows Update to be cached by ISPs. But it may simply be an lack of thought. Would probably help if some large ISPs tried to talk them to senses, but I suppose their preferred solution would be to simply set up an Akamai at the ISP..
Regards Henrik
