Search squid archive

Re: Squid with TPROXY slow response to requests

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



trasor wrote:
I am running squid with tproxy having followed the wiki Features/TPROXY in bridged mode. Squid is working properly with no evident errors in the logs, however initial requests are extremely slow to respond. i.e the first request for yahoo.com may take several seconds to retrieve the information and return a page to the requester, the second request is nearly instantaneous. And as long as the page is requested over and over the page will remain instantaneous. If the page is not requested for say longer than 45 seconds, the response time goes back to taking several seconds to respond. I believe this not to necessarily be a squid issue, but perhaps a routing issue as if I turn off the ebtables and iptables request/response times drastically improve. Has anyone else seen this phenomenon?

*server specs:*
Fedora 7 64bit
16GB ram
1.5TB raid 5 configured
2 10/100/1000 ethernet bridged

*squid -v:*

Squid Cache: Version 3.1.0.6
configure options: '--enable-linux-netfilter' '--enable-removal-policies=heap' 'enable-storeio=aufs' --with-squid=/usr/src/squid-3.1.0.6 --enable-ltdl-convienience

*squid.conf:*

visible_hostname site.domain.com acl manager proto cache_object
acl localhost src 127.0.0.1/32
acl to_localhost dst 127.0.0.0/8
acl localnet src X.X.X.X/20    # RFC1918 possible internal network
acl localnet src X.X.X.X/20    # RFC1918 possible internal network
acl localnet src X.X.X.X/21 # RFC1918 possible internal network
acl localnet src X.X.X.X/21 # RFC1918 possible internal network
acl SSL_ports port 443
acl Safe_ports port 80        # http
acl Safe_ports port 21        # ftp
acl Safe_ports port 443        # https
acl Safe_ports port 70        # gopher
acl Safe_ports port 210        # wais
acl Safe_ports port 1025-65535    # unregistered ports
acl Safe_ports port 280        # http-mgmt
acl Safe_ports port 488        # gss-http
acl Safe_ports port 591        # filemaker
acl Safe_ports port 777        # multiling http
acl CONNECT method CONNECT
http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localnet
http_access deny all
http_port 3128
http_port 3129 tproxy
hierarchy_stoplist cgi-bin ?
cache allow all
refresh_pattern ^ftp:        1440    20%    10080
refresh_pattern ^gopher:    1440    0%    1440
refresh_pattern -i (/cgi-bin/|\?) 0    0%    0
refresh_pattern .        0    20%    4320
#refresh_pattern ^ftp: 1440 20% 10080
#refresh_pattern ^gopher: 1440 0% 1440
#refresh_pattern -i \.(gif|png|jpg|jpeg|ico)$ 10080 90% 43200 override-expire ignore-no-cache ignore-no-store ignore-private ignore-reload reload-into-ims #refresh_pattern -i \.(iso|img|avi|wav|mp3|mp4|mpg|mpeg|swf|flv|x-flv|dll|do|xsp|wma|wmv|xml|asp|aspx)$ 43200 150% 432000 override-expire ignore-no-cache ignore-no-store ignore-private ignore-reload reload-into-ims #refresh_pattern -i \.(deb|rpm|exe|zip|tar|tgz|ram|rar|bin|ppt|doc|txt|tiff|pdf)$ 10080 200% 43200 override-expire ignore-no-cache ignore-no-store ignore-private ignore-reload reload-into-ims
#refresh_pattern -i \.index.(html|htm)$ 0 40% 10080 reload-into-ims
#refresh_pattern -i \.(html|htm|css|js|jsp|php|jtp|mspx|pl)$ 1440 40% 40320
#refresh_pattern . 0 40% 40320
cache_mem 10000 MB
memory_replacement_policy heap LFUDA
maximum_object_size 300 MB
coredump_dir /usr/local/squid/var/cache
access_log /var/logs/access.log squid
cache_log /var/logs/cache.log
cache_store_log none
cache_replacement_policy heap LFUDA
cache_dir aufs /cache 10000 16 256

*ebtables and iptables:*

ebtables -t broute -A BROUTING -i eth1 -p ipv4 --ip-proto tcp --ip-dport 80 -j redirect --redirect-target DROP ebtables -t broute -A BROUTING -i eth0 -p ipv4 --ip-proto tcp --ip-sport 80 -j redirect --redirect-target DROP

I'm a little fuzzy on when BROUTING takes place. DROP does not look that good though. Can you clarify what the above does in your understanding please?


Going by the packet flow map http://l7-filter.sourceforge.net/PacketFlow.png I would think the packets on a bridge device naturally flow along the bottom line of processing (blue). Which does include the iptables mangle PREROUTING table containing the TPROXY and DIVERT rules.

Have you tried it without the special ebtables rules, only the iptables rules?

There _may_ be some need for adding an ACCEPT for port 80 stuff in the ebtables filter INPUT/OUTPUT tables. But other than that I would expect the iptables mangle stuff to be all.

iptables -t mangle -N DIVERT
iptables -t mangle -A DIVERT -j MARK --set-mark 1
iptables -t mangle -A DIVERT -j ACCEPT
iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT
iptables -t mangle -A PREROUTING -i eth1 -p tcp --dport 80 -j TPROXY --tproxy-mark 0x1/0x1 --on-port 3129
echo 1 > /proc/sys/net/ipv4/ip_forward
ip rule add fwmark 1 lookup 100
ip route add local 0.0.0.0/0 dev lo table 100
cd /proc/sys/net/bridge/
for i in *
do
 echo 0 > $i
done
unset i


If running strictly bridged without ebtables or iptables, requests are processed at 'normal' speed but nothing is passed on to squid for cacheing. I went so far as running squid as a memory only cache and tried running with squid-3.1.0.8 with the same results. Any help speeding this up would be most appreciated.

Tom


Amos
--
Please be using
  Current Stable Squid 2.7.STABLE6 or 3.0.STABLE16
  Current Beta Squid 3.1.0.9

[Index of Archives]     [Linux Audio Users]     [Samba]     [Big List of Linux Books]     [Linux USB]     [Yosemite News]

  Powered by Linux