On 02/01/2013 03:52 PM, Eliezer Croitoru wrote:
On 2/1/2013 8:04 PM, Luciano Ruete wrote:
I've already tested and the above seams to be true. How can I know for
shure if there are or not objects in the cache_dir greater than
maximum_object_size_in_memory?
I can ran more tests if you give me further instructions, or can try a
patch if provided.
Regards.
We need a set of http cachable objects in different sizes:
tiny:
http://repo.ngtech.co.il/rpm/centos/6/x86_64/squid-debuginfo-3.2.5-1.el6.x86_64.rpm.asc
avg:
http://repo.ngtech.co.il/squid/cachecluster.png
avg download:
http://repo.ngtech.co.il/rpm/centos/6/others/openssh-6.1p1-81.el6.x86_64.rpm
big download:
http://repo.ngtech.co.il/rpm/centos/6/x86_64/squid-debuginfo-3.2.6-1.el6.x86_64.rpm
large download:
http://dl.fedoraproject.org/pub/fedora/linux/releases/18/Fedora/x86_64/iso/Fedora-18-x86_64-netinst.iso
very large download:
http://dl.fedoraproject.org/pub/fedora/linux/releases/18/Fedora/x86_64/iso/Fedora-18-x86_64-DVD.iso
All the above should have proper cache directive that can be checked
using:
http://redbot.org
(ignore my testing Link headers)
You can set these settings in squid.conf
store_dir_select_algorithm round-robin
#^^default is: "least-load" which can cause your problem.
cache_dir rock /var/spool/squid/rock 1000 min-size=1024 max-size=31000
max-swap-rate=250 swap-timeout=350
cache_dir aufs /var/spool/squid/aufs 30000 16 256 min-size=209715
max-size=734003200
maximum_object_size_in_memory 512 KB #default
minimum_object_size 0 KB #default
maximum_object_size 300 MB #non default
##end conf
First try only the round-robin to make sure this is not the basic
cause for problems.
reload\restart check the cache_dir sizes before and after getting into
a website like yahoo movies or any site with many objects in it.
With this conf, objects larger than 512KB does not get cache into AUFS
cache_dir, example:
requesting
1359747785.360 97216 127.0.0.1 TCP_MISS/200 9225893 GET
http://repo.ngtech.co.il/rpm/centos/6/x86_64/squid-debuginfo-3.2.6-1.el6.x86_64.rpm
- HIER_DIRECT/58.28.153.233 application/x-rpm
results in this in store log:
1359747785.360 RELEASE -1 FFFFFFFF E8A98EBEE591A2878F4795D16C3A434D 200
1357962518 1357827095 1389498518 application/x-rpm 9225364/9225364 GET
http://repo.ngtech.co.il/rpm/centos/6/x86_64/squid-debuginfo-3.2.6-1.el6.x86_64.rpm
And no change in AUFS size at all.
File with less than 512KB (and above 205K min-size) does goes to AUFS,
example
requesting
1359747696.788 8644 127.0.0.1 TCP_MISS/200 231025 GET
http://repo.ngtech.co.il/rpm/centos/6/others/openssh-6.1p1-81.el6.x86_64.rpm
- HIER_DIRECT/58.28.153.233 application/x-rpm
results in this in store log:
1359747696.784 SWAPOUT 01 00000000 9641F2C5678BA0B610DC647C16AE8EAF 200
1359744929 1357195218 1391280929 application/x-rpm 230420/230420 GET
http://repo.ngtech.co.il/rpm/centos/6/others/openssh-6.1p1-81.el6.x86_64.rpm
And AUFS dir size gets incremented acording.
ROCK storage also grows if small objects are requested.
Then try to change only the maximum_object_size_in_memory from 512
default to 8 KB which will cause almost no ram cache and mostly dir
cache to make sure that the reason for the low counters is not because
of mem caching.
Check the cache_dir before and after...
If i lower maximum_object_size_in_memory to 8 KB then
requesting:
1359749023.713 1683 127.0.0.1 TCP_MISS/200 1466 GET
http://repo.ngtech.co.il/rpm/centos/6/x86_64/squid-debuginfo-3.2.5-1.el6.x86_64.rpm.asc
- HIER_DIRECT/58.28.153.233 text/plain
1359749024.590 2561 127.0.0.1 TCP_MISS/200 33255 GET
http://repo.ngtech.co.il/squid/cachecluster.png -
HIER_DIRECT/58.28.153.233 image/png
1359749028.613 6583 127.0.0.1 TCP_MISS/200 231025 GET
http://repo.ngtech.co.il/rpm/centos/6/others/openssh-6.1p1-81.el6.x86_64.rpm
- HIER_DIRECT/58.28.153.233 application/x-rpm
1359749121.203 99173 127.0.0.1 TCP_MISS/200 9225894 GET
http://repo.ngtech.co.il/rpm/centos/6/x86_64/squid-debuginfo-3.2.6-1.el6.x86_64.rpm
- HIER_DIRECT/58.28.153.233 application/x-rpm
results in:
1359749023.712 SWAPOUT 00 000031E8 F104D7F44EDABE94F758EC16BD02ADA0 200
1359744786 -1 1360608786 text/plain 876/876 GET
http://repo.ngtech.co.il/rpm/centos/6/x86_64/squid-debuginfo-3.2.5-1.el6.x86_64.rpm.asc
1359749024.590 RELEASE -1 FFFFFFFF 5BFF306024844E48B8ED6A3BD89312FF 200
1359744790 -1 1360608790 image/png 32719/32719 GET
http://repo.ngtech.co.il/squid/cachecluster.png
1359749028.613 RELEASE -1 FFFFFFFF 45F6D81746DB2C8703359DEF1494F04D 200
1359744929 1357195218 1391280929 application/x-rpm 230420/230420 GET
http://repo.ngtech.co.il/rpm/centos/6/others/openssh-6.1p1-81.el6.x86_64.rpm
1359749121.203 RELEASE -1 FFFFFFFF 64FDA3C6D9598638709C7D5188D99793 200
1357962518 1357827095 1389498518 application/x-rpm 9225364/9225364 GET
http://repo.ngtech.co.il/rpm/centos/6/x86_64/squid-debuginfo-3.2.6-1.el6.x86_64.rpm
Only the smallest object goes to cache and if I flood the proxy with
user's traffic only the rock storage grows. And only object that are
less than 8k get a SWAPOUT in store_log.
If there is no straight answer if it's ok or not and if no clean
answer try to use all the directives I gave you together and try to
download each of the set of files I gave you.
The tiny should be cached in mem.
all the others should be cached in a cache_dir by the size.
the large ISO file should be cached in any case in the UFS cache_dir.
Feel free to ask me anything.
I am also in squid IRC channel here and there.
Let me know if you need any further test.
