On 2/1/2013 10:06 PM, Luciano Ruete wrote:
You can set these settings in squid.conf
store_dir_select_algorithm round-robin
#^^default is: "least-load" which can cause your problem.
cache_dir rock /var/spool/squid/rock 1000 min-size=1024 max-size=31000
max-swap-rate=250 swap-timeout=350
cache_dir aufs /var/spool/squid/aufs 30000 16 256 min-size=209715
max-size=734003200
maximum_object_size_in_memory 512 KB #default
minimum_object_size 0 KB #default
maximum_object_size 300 MB #non default
##end conf
First try only the round-robin to make sure this is not the basic
cause for problems.
reload\restart check the cache_dir sizes before and after getting into
a website like yahoo movies or any site with many objects in it.
With this conf, objects larger than 512KB does not get cache into AUFS
cache_dir, example:
requesting
1359747785.360 97216 127.0.0.1 TCP_MISS/200 9225893 GET
http://repo.ngtech.co.il/rpm/centos/6/x86_64/squid-debuginfo-3.2.6-1.el6.x86_64.rpm
- HIER_DIRECT/58.28.153.233 application/x-rpm
results in this in store log:
1359747785.360 RELEASE -1 FFFFFFFF E8A98EBEE591A2878F4795D16C3A434D 200
1357962518 1357827095 1389498518 application/x-rpm 9225364/9225364 GET
http://repo.ngtech.co.il/rpm/centos/6/x86_64/squid-debuginfo-3.2.6-1.el6.x86_64.rpm
And no change in AUFS size at all.
File with less than 512KB (and above 205K min-size) does goes to AUFS,
example
This is since they are cached in RAM cache until they dont need to be
swapped into disk.
The above is normal squid behavior.
requesting
1359747696.788 8644 127.0.0.1 TCP_MISS/200 231025 GET
http://repo.ngtech.co.il/rpm/centos/6/others/openssh-6.1p1-81.el6.x86_64.rpm
- HIER_DIRECT/58.28.153.233 application/x-rpm
results in this in store log:
1359747696.784 SWAPOUT 01 00000000 9641F2C5678BA0B610DC647C16AE8EAF 200
1359744929 1357195218 1391280929 application/x-rpm 230420/230420 GET
http://repo.ngtech.co.il/rpm/centos/6/others/openssh-6.1p1-81.el6.x86_64.rpm
And AUFS dir size gets incremented acording.
ROCK storage also grows if small objects are requested.
Good this is what it suppose to do.
Then try to change only the maximum_object_size_in_memory from 512
default to 8 KB which will cause almost no ram cache and mostly dir
cache to make sure that the reason for the low counters is not because
of mem caching.
Check the cache_dir before and after...
If i lower maximum_object_size_in_memory to 8 KB then
requesting:
1359749023.713 1683 127.0.0.1 TCP_MISS/200 1466 GET
http://repo.ngtech.co.il/rpm/centos/6/x86_64/squid-debuginfo-3.2.5-1.el6.x86_64.rpm.asc
- HIER_DIRECT/58.28.153.233 text/plain
1359749024.590 2561 127.0.0.1 TCP_MISS/200 33255 GET
http://repo.ngtech.co.il/squid/cachecluster.png -
HIER_DIRECT/58.28.153.233 image/png
1359749028.613 6583 127.0.0.1 TCP_MISS/200 231025 GET
http://repo.ngtech.co.il/rpm/centos/6/others/openssh-6.1p1-81.el6.x86_64.rpm
- HIER_DIRECT/58.28.153.233 application/x-rpm
1359749121.203 99173 127.0.0.1 TCP_MISS/200 9225894 GET
http://repo.ngtech.co.il/rpm/centos/6/x86_64/squid-debuginfo-3.2.6-1.el6.x86_64.rpm
- HIER_DIRECT/58.28.153.233 application/x-rpm
results in:
1359749023.712 SWAPOUT 00 000031E8 F104D7F44EDABE94F758EC16BD02ADA0 200
1359744786 -1 1360608786 text/plain 876/876 GET
http://repo.ngtech.co.il/rpm/centos/6/x86_64/squid-debuginfo-3.2.5-1.el6.x86_64.rpm.asc
1359749024.590 RELEASE -1 FFFFFFFF 5BFF306024844E48B8ED6A3BD89312FF 200
1359744790 -1 1360608790 image/png 32719/32719 GET
http://repo.ngtech.co.il/squid/cachecluster.png
1359749028.613 RELEASE -1 FFFFFFFF 45F6D81746DB2C8703359DEF1494F04D 200
1359744929 1357195218 1391280929 application/x-rpm 230420/230420 GET
http://repo.ngtech.co.il/rpm/centos/6/others/openssh-6.1p1-81.el6.x86_64.rpm
1359749121.203 RELEASE -1 FFFFFFFF 64FDA3C6D9598638709C7D5188D99793 200
1357962518 1357827095 1389498518 application/x-rpm 9225364/9225364 GET
http://repo.ngtech.co.il/rpm/centos/6/x86_64/squid-debuginfo-3.2.6-1.el6.x86_64.rpm
Only the smallest object goes to cache and if I flood the proxy with
user's traffic only the rock storage grows. And only object that are
less than 8k get a SWAPOUT in store_log.
OK and in this exact scenario with weither 64KB and 512KB everything is
fine and I think it might be because of something with the buffer but
i'm not sure about it yet.
If there is a bug this is the only one exists and only in specific cases
which a wrong tuning of squid.conf is being done.
If there is no straight answer if it's ok or not and if no clean
answer try to use all the directives I gave you together and try to
download each of the set of files I gave you.
The tiny should be cached in mem.
all the others should be cached in a cache_dir by the size.
the large ISO file should be cached in any case in the UFS cache_dir.
Feel free to ask me anything.
I am also in squid IRC channel here and there.
Let me know if you need any further test.
Just use the defaults and change to RoundRobin selection.
Dont change any object size default else then maximum_object_size to 700 MB
If you have any visible problem after that file a bug at the bugzilla
and also refer to this mail as part of the process.
http://bugs.squid-cache.org/
--
Eliezer Croitoru
http://www1.ngtech.co.il
