On 10/10/2013 1:00 a.m., Dash Four wrote:
Amos Jeffries wrote:
On 9/10/2013 9:39 a.m., Dash Four wrote:
I have the following problem: I use the "hosts" file to store static
address mappings, usually containing sites which use geo address
mapping (in other words, determine the ip address one is going to
use depending on the geographic region the request is made from).
One such site is youtube for example.
Squid caches web pages content from these sites, but when I change
the IP address in "hosts", squid is still using the old IP address
for some reason, even though I verified that the new mapping is used
by the system.
In such cases, I have to stop squid, wipe out the entire cache and
then restart squid in order for the new host->ip mapping to be used.
This can't be right!
What I presume is happening is that somewhere along the line, squid
is storing clear ip addresses in the cache (probably to speed up
serving page content requests) and then it does not check whether
that ip address has changed. If that is so, could this be fixed,
because the idea of me clearing the entire cache every time I change
one of the "hosts" mappings does not exactly appeal to me. Thanks.
Sort of. Squid loads the hosts file contents into DNS result cache
with an infinite TTL in order to provide a single lookup mechanism
for any DNS entry and prevent remote lokups of those FQDN. These are
only loaded on startup and reconfigure time.
Thanks Amos, but I am not sure what you describe above is actually
happening.
When the "hosts" file is updated, I tried to reload squid's
configuration (-k reconfigure), but that did not help as squid was
still trying to connect to the "old" ip address. I then stopped (-k
shutdown) and then started squid without touching the existing cache -
again, squid was still referring to the old host->ip mapping.
This is not possible in Squid DNS resolver. Squid ipcache is held only
in RAM. When you stop the process and restart it a whole new area of RAM
gets allocated and the ipcache starts from empty.
You can verify this using the "ipcache" cachemgr report, which lists all
the host->IP mappings known to Squid. After a restart the only entries
will be those freshly loaded from hosts file, names resovled while
loading the configuration file, and the newly resolved hostnames from
any traffic between sartup and your report fetch.
The only way I "fixed" this is when I shutdown squid, delete the
entire cache and then start it - only then the correct host->ip
mapping was used, which would suggests that squid keeps the "old" ip
addresses somewhere, I presume in the cache. I should also say this:
the incorrect host->ip mapping is (mainly) used to retrieve secondary
links (not main page urls), which appear as links on the main web pages.
I initially thought that my web browser cache is to blame, but this
was eliminated as a possibility since I manually wiped out the entire
web browser cache and restarted the browser and still got the "old"
mapping.
Squid at this time does not yet have any mechanism for watching the
file for changes (patches welcome!).
If your OS supports a filesystem trigger feature such as inotify or
dnotify you can script up a restart of Squid when hosts changes.
Yes, inotify is fully supported, but my issue is with squid caching
dns requests - that should not happen, or, at the very least, I should
be able to disable this behaviour. DNS-caching is the job of the DNS
server or, as in my case, the dnsmasq program I am using as dns
resolver, not squid.
DNS has the same client/proxy/origin server model as HTTP, with the same
caching hierarchy model on a simpler scale. Authoritative resolvers are
the origins in DNS. Recursive resolvers are the proxies in DNS. And
software like Squid internal resolver are the "browser" equivalent -
with their own data cache.
Squid internal resolver is a standards compliant (RFC 1035, 2181,
2874, 3226, 3596, 6762) DNS client and obeys the storage TTL associated
with each DNS record it has cached.
Amos
