Search squid archive

Re: Re: slow browsing in centos 6.3 with squid 3 !!

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 24/02/2013 2:26 a.m., Ahmad wrote:
Hi mr Amos ,
ive installed libcap2 already in centos 6.3
[root@squid ~]# yum install libcap
Loaded plugins: fastestmirror, security
Loading mirror speeds from cached hostfile
  * base: ftp.hosteurope.de
  * extras: ftp.hosteurope.de
  * updates: ftp.hosteurope.de
base
| 3.7 kB     00:00
extras
| 3.5 kB     00:00
updates
| 3.5 kB     00:00
Setting up Install Process
Package libcap-2.16-5.5.el6.x86_64 already installed and latest version
Nothing to do
[root@squid ~]#

==================================
do u mean i  need to set option while comoiling squid ??

Sorry, I was not clear. You need the developer version of each dependent library installed to build support into Squid. Not just the run-time version.
The package for that library is probably called libcap-dev or libcap-2-dev



i have another question :
i have alot of errors after i loaded about 1000 users with 130 M to squid :
here is it from cache.log

#############################
{Accept: */*
Content-Type: application/x-www-form-urlencoded
2013/02/23 08:20:29.001| WARNING: HTTP header contains NULL characters
{Accept: */*
Content-Type: application/x-www-form-urlencoded}
NULL
{Accept: */*
Content-Type: application/x-www-form-urlencoded

One of your clients is sending very broken headers. Not only does it contain a 0x00 byte after the Content-Type, but the headers are then repeated at least once more with the same 0x00 byte after the repeated Content-type line.

In 3.2+ you can use debug_options 11,2 to retrieve a trace of the HTTP headers with full details of which client sent them and which site is involved. Please report this problem to whomever is relevant as the cause.

<snip>
################################################
here is my squid.conf file after some preparing :
[root@squid ~]# cat /etc/squid/squid.conf | less
#
#
#
#
###################
acl all src all
acl manager proto cache_object
acl localnet src 192.168.1.0/24 x.x.0.0/16 x.x.0.0/16
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8

The above "all", "manager", "localhost", "to_localhost" ACLs are all built-in nowdays. For your 3.1 Squid you an remove the "all" ACL to avoid some startup warnings. For your 3.3 Squid you should also remove the others (except localnet) from your config file. Particualrly the manager ACL type has changed so leaving it present will cause startup failures in 3.2+.

acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 590         # filemaker
acl Safe_ports port 777         # multiling http
acl CONNECT method CONNECT
#Default:\
visible_hostname squid

The visible_hostname n 3.2+ is used in URLs to load the icons and error page CSS etc which is served straight out of Squid using its hostname as the public FQDN in the URL. For example: with the above setting your Squid error pages will contain <img src="http://squid/squid-internal-icons/SN.png";>

Squid is pretty good now at detecting the /etc/hostname of the machine it is running on. You should consider registering that machine hostname in DNS so that both forward and reverse DNS lookups will resolve it. Then the URLs will work using the hostname and you do not have to manually configure it in Squid.
 Alternatively please consider using a FQDN as visible_hostname.


coredump_dir /var/spool/squid
####squidguard###################
redirect_program /usr/local/bin/squidGuard -c
/usr/local/squidGuard/squidGuard.conf
redirect_children 300

The redirect_* directives have been renamed in Squid-3 to match what they actually do. Replace the "redirect_" part of those names with "url_rewrite_" to avoid confusion and more WARNINGS in future. There is no operational difference between the directives at present, just less confusion.

cache_effective_user squid
cache_effective_group squid
##############################
#Recommended minimum configuration:
#
# Only allow cachemgr access from localhost
http_access allow manager localhost
http_access allow localnet

NP: the above line allows any localnet machines to use manager requests. eg http://squid-machine/squid-internal-mgr/info

http_access deny manager
# Deny requests to unknown ports
http_access deny !Safe_ports
# Deny CONNECT to other than SSL ports
http_access deny CONNECT !SSL_ports
# And finally deny all other access to this proxy
http_access deny all
#Allow ICP queries from everyone
icp_access allow all
hierarchy_stoplist cgi-bin ?

hierarchy_stoplist is pretty useless in curret Squid. It is scheduled for removal in later releases. You can drop it from your config.

#####################
#######################################
access_log /var/log/squid/access.log
cache_dir aufs /cache1 500000 32 256
cache_dir aufs /cache2 500000 32 256
cache_dir aufs /cache3 500000 32 256
cache_mem 2000 MB
#access_log none
#cache_store_log none
##########################
http_port 127.0.0.1:3128
http_port x.x.x.x:65000
http_port 3128
http_port 3129 tproxy
########### Performance Related Config:
hierarchy_stoplist cgi-bin ?
hierarchy_stoplist is not performance related and this entry is a duplicate of the above one.

forwarded_for off
forwarded_for is also not performance related. The setting above is anonymizing the end-clients IP addresses and making Squid insert "X-Forwarded-For: unknown" into the relayed headers.

The settings available on that directive have changed in 3.2. On your 3.3 squid you should re-consider its usage... * If your intent was to hide Squid from visibility on the pathway (transparent HTTP proxy), then set "forwarded_for transparent". * If your intent was to anonymize the client IP addresses and make Squid visible as the client, then set "forwarded_for delete"

.. However I notice you have a setting further down which undoes this disabling and re-enables the default Squid operations. So maybe what you intended was to omit the directive entirely.

half_closed_clients off
relaxed_header_parser on

Both of the above settings are the default values for Squid-3. You can simplify your config by removing the above two lines completely.

reload_into_ims on
quick_abort_min 0 KB
quick_abort_max 0 KB
client_lifetime 15 minutes
read_timeout 5 minutes
request_timeout 1 minutes
ie_refresh on
You have many users with IE 5.5 or older? If not consider disabling this again to reduce the amount of browser sniffing Squid has to perform.

vary_ignore_expire on
##########################################
maximum_object_size_in_memory 64 KB

NP: Squid-3 have a much better memory cache implementation than Squid-2. You may want to reconsider this limit. Check your stored response size statistics
memory_replacement_policy heap GDSF
cache_replacement_policy heap LFUDA
###########################################
ipcache_size 2048
ipcache_low 98
ipcache_high 99
memory_pools off
pipeline_prefetch on
############################################
httpd_suppress_version_string on
server_persistent_connections on
client_persistent_connections on
pconn_timeout 2 minutes
persistent_request_timeout 1 minute
###########################################
#### drx
#via off
visible_hostname squid
The above is a duplicate of the visible_hostname being set up the top.

#memory_cache_shared off
########### WCCP2 Config:
wccp2_router x.x.x.x
wccp_version 2
wccp2_forwarding_method 2
wccp2_return_method 2
#wccp2_assignment_method mask
wccp2_service dynamic 80
wccp2_service_info 80 protocol=tcp flags=src_ip_hash priority=240 ports=80
wccp2_service dynamic 90
wccp2_service_info 90 protocol=tcp flags=dst_ip_hash,ports_source
priority=240 ports=80
##########################################
###########################################
#default option
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320
#################################################
hierarchy_stoplist cgi-bin ?
This is a duplicate of the stoplist setting up above. You can remove it.

forwarded_for on

This setting contradicts and overrides the forwarded_for setting above. The result is that Squid is back doing its default operations again.

half_closed_clients off

Another duplicate setting. You can remove the above line completely.

#persistent_request_timeout 2 minutes
max_filedescriptors 65536
max_open_disk_fds 65536
relaxed_header_parser on
reload_into_ims on
client_lifetime 15 minutes
read_timeout 5 minutes
request_timeout 1 minutes
#extension_methods NICK
ie_refresh on
ignore_expect_100 on
vary_ignore_expire on
###############################
################################
httpd_suppress_version_string on
server_persistent_connections on
client_persistent_connections on
pconn_timeout 2 minutes
persistent_request_timeout 1 minute
shutdown_lifetime 20 seconds
#############################
cache_swap_low 98
cache_swap_high 99
cache_replacement_policy heap LFUDA
#request_header_max_size 2048 KB
#minimum_object_size 512 bytes
#maximum_object_size 5 GB
minimum_object_size 0
maximum_object_size 135128 kB

The unit type on these diectives is case-sensitive. The "kB" should be "KB".
 Or possibly the setting was supposed to be "maximum_object_size 132 MB" ?

Note that setting this low limits your Squid to dropping any objects >132MB and always MISS'ing for them.

PS. For organization I'd shuffle these all up next to the cache_dir lines so the storage-related stuff is all together in the config file.

negative_ttl 0 seconds

This is the default negative_ttl value for all Squid-3.1+. You can remove the above line from your config.

###############################
fstab file :
[root@squid ~]# cat /etc/fstab

#
# /etc/fstab
# Created by anaconda on Thu Feb 21 10:05:33 2013
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=7e8424aa-d3a7-4132-9bed-a40ac8400bb3 /                       ext4
defaults        1 1
tmpfs                   /dev/shm                tmpfs   defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0

/dev/sdb /cache1 ext4 noatime,barrier=0,data=writeback,commit=100 0 2
/dev/sdd /cache2 ext4 noatime,barrier=0,data=writeback,commit=100 0 2
/dev/sde /cache3 ext4 noatime,barrier=0,data=writeback,commit=100 0 2

=============================

also i note that there is a low speed in writing on my harddrives !!!!
[root@squid ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1             165G  145G   12G  93% /
tmpfs                  16G     0   16G   0% /dev/shm
/dev/sdb              550G  1.2G  521G   1% /cache1
/dev/sdd              550G  1.2G  521G   1% /cache2
/dev/sde              550G  1.3G  521G   1% /cache3
[root@squid ~]#

Speed? That looks like size of cached data on disk to me. It starts off low and grows over time. That is ~12MB of cached objects after how long run-time? at what HIT ratio? If you are getting a lot of MISS traffic on the 3.3 Squid you can try updating to the daily snapshot package which has a few object size bugs fixed.

Amos


[Index of Archives]     [Linux Audio Users]     [Samba]     [Big List of Linux Books]     [Linux USB]     [Yosemite News]

  Powered by Linux