On 19/09/17 23:54, Iraj Norouzi wrote:
hi Antony
thanks for you reply
i setup squid on ubuntu and centos
Why both?
because of test and because i not get the result
with tproxy and wccp for 6 gb/s traffic
What hardware are you using for that sort of traffic flow?
i use hp DL360 with 2 6 core processor with 3 GHZ and 64 GB RAM and 1 TB HDD
but when i try to test squid with 40 mb/s traffic
How are you generating "40 mb/s traffic"? I'm assuming that your Internet
connection is 6Gbps as stated above, so how are you restricting this down to
40Mbps for testing?
i redirect one class of ip address with 40 mb/s traffic for test of
squid and i am going decide to redirect whole of traffic to squid after
getting fast browsing
Squid is designed to optimize and reduce *bandwidth*. "fast browsing" is
just a nice side effect of caching. It is a mistake to think that Squid
will always produce faster browsing.
This is especially true during the initial cache warm-up period where
DNS and HTTP objects are being fetched for the first time. Speed can
only come from future fetches being reduced by the cache.
So when you are testing for speed with real traffic make sure there has
been at least a few hrs for the caches to warm up.
Since you are testing with RAM-only caching right now be aware that
every time you restart Squid *all* its caches (for all data types) get
erased back to "cold"/empty.
it response very slow
Numbers please.
websites load at 2 or 1 second by direct browsing and load at 10 second
or not load by squid
while when i use direct browsing i can browse websites very fast
Is the direct traffic still being routed through the Squid server (you say
you're using tproxy, so I assume this is an intercept machine with the
traffic
going through it between client and server)?
no, HTTP traffic redirect to squid by wccp and access-list config on Cisco
Cisco is tunneling the packets with WCCP to the Squid machine, which is
intercepting the traffic with TPROXY.
So actually "yes", if not something would be terribly broken in your
WCCP and TPROXY setup.
ip wccp 80 redirect-list wccp
ip wccp 90 redirect-list wccp_to_inside
ip access-list extended wccp
remark Permit http access from clients
permit tcp x.x.x.x 0.0.0.255 any eq www
deny ip any any
ip access-list extended wccp_to_inside
permit tcp any eq www x.x.x.x 0.0.0.255
deny ip any any
i used tcpdump for tracing connections arrive time and there was no problem,
Arrival time where? From the origin server to Squid? From Squid to the
client? What are you actually measuring?
yes, arriving source packets from clients to squid interface,by time
that i push enter on browser address bar and getting packets on tcpdump
immediately
Packets arriving at Squid from the client is only the first ~1% of
things that are going on. It would be a big problem if they took
anything more than a few ms to arrive.
Once the packets arrive there are DNS lookups to do. Delay in DNS
lookups is the most common cause of overall delays.
Then there is HTTP processing to find a source for the response. If the
request has never been seen before that means a fair amount of logic to
select potential upstream servers and attempt connections to them (maybe
several or even all of them).
Then the server request has to be generated, and wait for a response.
Only when that server response comes back can stuff for the client
response start to happen.
When the cache is involved the total time could be under 1ms, or
somewhat around 50ms. If there are lots of server things to do the time
can be hundreds of ms.
i used watch -d for tracing packets match by iptables rules and it was ok,
Please be more specific - what did you measure and what does "OK" mean?
i add rule to iptables for tracing one website packets and i saw them on
kern.log that matched with the rule, so if you need please tell me to
send commands that i used.
Which packets. As I detailed above, there are a minimum of 2 TCP
connections involved with delivering a MISS object (client->Squid, and
Squid->server) potentially many more if there are network issues
connecting to server(s).
ip rule add fwmark 1 lookup 100 ip route add local 0.0.0.0/0
<http://0.0.0.0/0> dev enp3s0f0 table 100
iptables -t mangle -A PREROUTING -d $LOCALIP -j ACCEPT
You also need a rule before the above one which blocks traffic to
"-d $LOCALIP -p 3129 -j REJECT"
iptables -t
mangle -N DIVERT iptables -t mangle -A DIVERT -j MARK --set-mark 1
iptables -t mangle -A DIVERT -j ACCEPT iptables -t mangle -A PREROUTING
-p tcp -m socket -j DIVERT iptables -t mangle -A PREROUTING -p tcp
--dport 80 -j TPROXY --tproxy-mark 0x1/0x1 --on-port 3129
watch -d iptables -t mangle -vnL
Did you compare with and without Squid in place to see what differs?
no, as i told when i browsing directly it is well and packets not coming
to squid and exit from cisco to internet and arrive to clients from
cisco and because of traffic on cisco i can't enable debugging on it but
when i browse from squid i get latency so i suppose the problem is squid
or server that squid running on it
Your test is incomplete.
To eliminate WCCP and routing issues being the cause of problems you
need to compare:
A) normal browsing without WCCP or TPROXY.
B) normal browsing with WCCP tunneling traffic to the Squid machine (no
Squid running and no TPROXY).
C) normal browsing with WCCP tunneling traffic to the Squid machine
where TPROXY diverts the traffic into Squid.
If things work any slower at test (B) than test (A) your problem is the
WCCP + Squid machine setup, not Squid itself.
i also used iptables trace command for tracing matching iptables rules,
there was no problem except i had latency on arriving packets on iptables
rule while tcpdump captured packets fast, it happened when my browsing was
so slow, at some times that my browsing was fast there was no latency on
iptables trace log.
That description is too vague to know exactly what you were measuring
and what
results you got.
iptables -t raw -A PREROUTING -s x.x.x.x -j TRACE
iptables -t raw -A OUTPUT -s x.x.x.x -j TRACE
tailf /var/log/kern.log
tcpdump -e -i enp3s0f0 -d x.x.x.x dst port 80
tcpdump -e -i enp3s0f0 -s x.x.x.x src port 80
"and what results you got" ?
i also used tcp and linux enhancement configurations
Details?
net.core.wmem_default=524288
net.core.wmem_max=16777216
net.core.rmem_default=524288
net.core.rmem_max=16777216
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_rmem = 66560 524288 16777216
net.ipv4.tcp_wmem = 66560 524288 16777216
net.core.somaxconn=4000
net.ipv4.tcp_timestamps=0
net.ipv4.tcp_sack=0
net.ipv4.tcp_fin_timeout=20
net.ipv4.ip_local_port_range=10240 65000
net.ipv4.tcp_keepalive_time = 900
net.ipv4.tcp_keepalive_intvl = 900
net.ipv4.tcp_keepalive_probes = 9
net.core.somaxconn = 5000
net.core.netdev_max_backlog = 8000
net.ipv4.tcp_max_syn_backlog = 8096
net.ipv4.tcp_slow_start_after_idle = 0
net.ipv4.tcp_tw_reuse = 1
Okay, so no improvement with those means they were not useful, please
ensure they are back at defaults so as not to add problems to other
experimental changes.
but nothing happened.
wccp send packets very well and tcpdump show capturing packets too but
browsing with squid is very slow.
Firstly, please define "slow" - do you mean it takes a long time for new web
pages / images / etc to appear (but once they start, they arrive quickly)
>> Iraj:
browse in 10 second or not browsing, webpages that i browse for first
time or i browse for multiple times
For that sort of delay:
* check the DNS response times for lookups made by the Squid machine.
The details of Squid's lookups can be found in the "idns" cache manager
report.
*
>> Antony:
, or do you mean that a continuous stream of data (a "download") arrives
more
slowly when going through Squid than going direct (and if so, what are the
different speeds)?
>> Iraj:
no just browsing,
>> Antony:
Secondly, what are you trying to achieve with Squid - what is its purpose in
your network?
>> Iraj:
caching pages and their objects for better internet browsing by clients
and save bandwith.
squid.conf
dns_v4_first on
acl fanava_net src x.x.x.x/32
acl Safe_ports port 80# http
acl Safe_ports port 280# http-mgmt
acl Safe_ports port 488# gss-http
acl Safe_ports port 777# multiling http
acl CONNECT method CONNECT
http_access allow localhost manager
http_access deny manager
http_access allow localhost
http_access allow fanava_net
http_access deny all
http_port 0.0.0.0:3128
http_port 0.0.0.0:3129 tproxy
cache_mem 50 GB
Does this machine actually have at least 52 GB of free RAM for Squid to use?
maximum_object_size_in_memory 100 MB
minimum_object_size 2 KB
maximum_object_size 6 GB
#cache_dir ufs /var/spool/squid 1024000 256 512
cache_swap_low 90
cache_swap_high 92
coredump_dir /var/spool/squid
# Image files
refresh_pattern -i \.(png|gif|jpg|jpeg|bmp|tif|tiff)$ 10080 90% 43200
# Compressed files
refresh_pattern -i \.(zip|rar|tar|gz|tgz|z|arj|lha|lzh|iso|deb|rpm)$
10080 90% 43200
# Binary files
refresh_pattern -i \.(exe|msi)$ 10080 90% 43200
# Multimedia files
refresh_pattern -i
\.(mp3|wav|mid|midi|ram|avi|wmv|mpg|mpeg|mp4|swf|flv|)$ 10080 90% 43200
# Document files
refresh_pattern -i \.(pdf|ps|doc|ppt|xls|pps)$ 10080 90% 43200
# HTML patterns
refresh_pattern -i \.index.(html|htm)$ 0 40% 10080
refresh_pattern -i \.default.(html|htm)$ 0 40% 10080
refresh_pattern -i \.(html|htm|css|js)$ 1440 40% 40320
# Default patterns
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320 # Fanava First
clients range to cache
refresh_pattern (Release|Packages(.gz)*)$ 0 20% 2880
cache_mgr ******
wccp2_router x.x.x.x
wccp_version 2
wccp2_rebuild_wait on
wccp2_forwarding_method 2
wccp2_return_method 2
wccp2_service dynamic 80
wccp2_service_info 80 protocol=tcp flags=src_ip_hash priority=240 ports=80
wccp2_service dynamic 90
wccp2_service_info 90 protocol=tcp flags=dst_ip_hash,ports_source
priority=240 ports=80
*Regards,
Iraj Norouzi*
Amos
_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users
