these were just samples of URLs ... ALL traffic is not being cached ...
I am talking about 24 hours of traffic on a busy network. If i switch
back to the old transparent setup everything works in fine. I am at a
loss why tproxy is broken
Amos Jeffries wrote:
Jamie Orzechowski wrote:
I have made the recommended changes but still everything is a
TCP_MISS and going DIRECT ... I removed the direct-urls for now just
in case. Any further suggestions?
<snip config: it does permit caching **if allowed**>
http://www.google.com/calendar/feeds/8vm1eh2d7rcltngl0omsa4qc3g%40group.calendar.google.com/private/full?
- DIRECT/208.67.217.230 text/html
http://www.google.com/calendar/feeds/8vm1eh2d7rcltngl0omsa4qc3g%40group.calendar.google.com/private/full?
- DIRECT/208.67.217.230 text/xml
1236865766.338 991 66.78.98.25 TCP_MISS/200 33924 GET
http://www.maximumsignal.ca/store/product.php? -
DIRECT/209.200.251.37 text/html
1236865766.993 546 66.78.98.25 TCP_MISS/200 2435 GET
http://www.maximumsignal.ca/store/antibot_image.php? -
DIRECT/209.200.251.37 image/png
Well, thanks for these. I ran a few of the pages through the
cacheability engine:
http://www.ircache.net/cgi-bin/cacheability.py
And got this most of the time:
Expires 606 weeks 5 days ago (Sat, 26 Jul 1997 05:00:00 GMT)
Cache-Control post-check=0, must-revalidate, no-store,
no-cache, pre-check=0
Last-Modified now (Fri, 13 Mar 2009 01:08:54 GMT)
"
* This object has been deliberately marked stale.
* The object had changed when validation was attempted.
* It will be revalidated on every hit, because it has a
Cache-Control: no-cache header.
* It won't be cached at all, because it has a Cache-Control: no-store
header.
* Because of the must-revalidate header, all caches will strictly
adhere to any freshness information you set.
* It doesn't have a Content-Length header present, so it can't be
used in a HTTP/1.0 persistent connection.
"
Sometimes they also have:
"
* This object requests that a Cookie be set; this makes it and other
pages affected automatically stale; clients must check them upon every
request.
"
Only the images seem to be cachable and only by cache with reasonable
refresh_patterns set.
Amos
Amos Jeffries wrote:
Here is the config ... it does work fine in "transparent" mode just
not
tproxy mode
acl manager proto cache_object
acl localhost src 127.0.0.1/32
acl to_localhost dst 127.0.0.0/8
acl localnet src 66.78.96.0/19
acl localnet src 64.235.192.0/19
acl localnet src 72.0.192.0/19
acl localnet src 192.168.1.0/24
acl localnet src 192.168.254.0/24
acl QUERY urlpath_regex cgi-bin \?
cache deny QUERY
You can drop the above. It's handled by the refresh_patterns.
hierarchy_stoplist cgi-bin ?
acl directurls url_regex -i "/etc/squid3/direct-urls"
cache deny directurls
You are not testing with one of these URLs are you?
cache deny localnet
This is definately causing MISS. Your http_access rules below prevent
non-localnet access. Therefore every received request will hit this
cache
prevention.
You want "cache allow all" at the end of the cache list.
always_direct allow directurls
always_direct allow localnet
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT
http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny to_localhost
http_access allow localnet
http_access allow localhost
http_access deny all
icp_access allow localnet
htcp_access allow localnet
icp_access deny all
htcp_access deny all
htcp_clr_access deny all
ident_lookup_access deny all
http_port 66.78.102.2:3128
http_port 66.78.102.2:3129 tproxy
cache_mgr support@xxxxxxxxxx
acl snmp snmp_community s64hf2
snmp_access allow snmp all
snmp_port 3401
snmp_incoming_address 192.168.1.8
snmp_outgoing_address 192.168.1.8
shutdown_lifetime 10 seconds
pid_filename /var/run/squid3.pid
mime_table /usr/share/squid3/mime.conf
icon_directory /usr/share/squid3/icons
error_directory /usr/share/squid3/errors/en
:( I would recommend removing that for a better experience by your
users.
(allows errors to appear in the visitors own language)
cache_effective_user proxy
ignore_unknown_nameservers on
dns_nameservers 66.78.99.4 66.78.99.5
<snip>
refresh_pattern -i \.index.(html|htm)$ 0 40% 10080
Double check this pattern you have files ending in:
'.index' any-character 'htm' optional-'l'
I think you maybe meant:
-i .index\.(html|htm)$
refresh_pattern -i \.(html|htm|css|js)$ 1440 40% 40320
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 40% 40320
Amos Jeffries wrote:
I am using squid 3.1.0.6
If I check the disk free while the cache is running I do not see
any of
my cache directories incrementing at all.
Any ideas?
Okay, so much for the easy answer. We will have to see your config to
tell
why its not caching.
Amos
Amos Jeffries wrote:
Jamie Orzechowski wrote:
Hi,
My post does not seem to be going to the list. Wondering if
you have
any ideas?
I think I have TPROXY working but running into some issues.
Checking my logs all my traffic shows up as a TCP_MISS
Squid 3.1.0.3? it has a storage problem that can show like this.
1236698452.579 79 66.78.98.194 TCP_MISS/200 542 GET
http://l1.zedo.com//log/p.gif? - DIRECT/72.247.244.10 image/gif
1236698452.634 293 66.78.98.194 TCP_MISS/200 4972 GET
http://blstb.msn.com/i/9B/DDD13A38CB8B34F4DFA3F7BFFF71.jpg -
DIRECT/192.221.114.124 image/jpeg
1236698452.878 100 66.78.98.194 TCP_MISS/200 1076 GET
http://h.foxsports.com/HG? - DIRECT/64.154.81.231 image/gif
1236698453.367 252 66.78.98.194 TCP_MISS/200 1368 GET
http://www.myinternetservices.com/live/visitor/index.php? -
DIRECT/72.232.167.111 image/gif
1236698454.087 13 66.78.98.194 TCP_MISS/200 812 GET
http://weyedata.pelmorex.com/WeatherEye/ObsData/CAON0090.xml -
DIRECT/207.96.160.37 text/xml
1236698455.251 116 66.78.98.194 TCP_MISS/200 1368 GET
http://www.myinternetservices.com/live/visitor/index.php? -
DIRECT/72.232.167.111 image/gif
1236698456.570 6451 66.78.98.194 TCP_MISS/200 45898 GET
http://www.facebook.com/profile.php? - DIRECT/69.63.176.140
text/html
1236698456.876 77 66.78.98.194 TCP_MISS/200 2765 GET
http://profile.ak.facebook.com/v227/2005/50/q638320646_36.jpg -
DIRECT/209.170.91.178 image/jpeg
My iptables is the following
echo 1 > /proc/sys/net/ipv4/ip_forward
/sbin/iptables -t mangle -N DIVERT
/sbin/iptables -t mangle -A DIVERT -j MARK --set-mark 1
/sbin/iptables -t mangle -A DIVERT -j ACCEPT
/sbin/iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT
/sbin/iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY
--tproxy-mark 0x1/0x1 --on-port 3129
//
any idea why I am not getting any TCP_HITS? ...
Amos
--
=-=-=-=-=-=-=-=-=-=-=-=-=
Jamie Orzechowski - CCNA
RipNET Ltd. System/Network Administrator
Tel.: 613-342-3946 x294
THIS MESSAGE IS INTENDED ONLY FOR THE ADDRESSEE,
IT MAY CONTAIN PRIVILEGED OR CONFIDENTIAL INFORMATION.
ANY UNAUTHORIZED DISCLOSURE IS STRICTLY PROHIBITED.
IF YOU HAVE RECEIVED THIS MESSAGE IN ERROR,
PLEASE NOTIFY ME IMMEDIATELY SO THAT I MAY CORRECT MY
INTERNAL RECORDS. PLEASE THEN DELETE THE ORIGINAL MESSAGE.
=-=-=-=-=-=-=-=-=-=-=-=-=
