Search squid archive

Re: rock issue

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



dear alex,

Was this 7% measured with max-size=32000 or without? [ i did not use max-size option]

When using AUFS (without rock), do you limit disk-cached object sizes to
~32KB (max-size=32000)? 
[ i use maximum_object_size_in_memory 250 MB and maximum_object_size 2 GB] // which i also use it in rock ]


squid version [ Version 4.8]
 thank u

From: Alex Rousskov <rousskov@xxxxxxxxxxxxxxxxxxxxxxx>
Sent: Tuesday, July 7, 2020 4:32 PM
To: patrick mkhael <patrick.mkhael@xxxxxxxxxxx>; squid-users@xxxxxxxxxxxxxxxxxxxxx <squid-users@xxxxxxxxxxxxxxxxxxxxx>
Subject: Re: [squid-users] rock issue
 
On 7/7/20 6:26 AM, patrick mkhael wrote:

> **What kind of hit ratio do you get with rock if you do not
> limit swap-rate and do not specify swap-timeout? [i also removed the max
> size as recomended], the gain ratio is max 13 %.

Noted, thank you.


> ​**What kind of hit ratio do you get with rock if you use one worker, one
> rock cache_dir, do not limit swap-rate, do not specify swap-timeout, and
> start Squid with -N to disable SMP? [ as recomended, only one rock
> cache_dir , no limit swap and excuted with -N option,the gain ration is 7%]

 


When using AUFS (without rock), do you limit disk-cached object sizes to
~32KB (max-size=32000)?


Finally, what is your Squid version?


Thank you,

Alex.


> ------------------------------------------------------------------------
> *From:* Alex Rousskov <rousskov@xxxxxxxxxxxxxxxxxxxxxxx>
> *Sent:* Saturday, July 4, 2020 3:40 AM
> *To:* patrick mkhael <patrick.mkhael@xxxxxxxxxxx>;
> squid-users@xxxxxxxxxxxxxxxxxxxxx <squid-users@xxxxxxxxxxxxxxxxxxxxx>
> *Subject:* Re: [squid-users] rock issue
>  
> On 7/3/20 4:50 AM, patrick mkhael wrote:
>
>> workers 3
>> cpu_affinity_map process_numbers=1,2,3,4,5,6 cores=1,2,3,4,5,6
>> cache_dir rock /rock1 200000 max-size=32000 swap-timeout=300 max-swap-rate=100
>> cache_dir rock /rock2 200000 max-size=32000 max-swap-rate=100 swap-timeout=300
>> cache_dir rock /rock3 200000 max-size=32000 max-swap-rate=100 swap-timeout=300
>> cache_mem 17 GB
>> maximum_object_size_in_memory 25 MB
>> maximum_object_size 1 GB
>> cache_miss_revalidate off
>> quick_abort_pct 95
>
>
> FYI: The combination of 1GB maximum_object_size and much smaller size
> limits for objects in memory and disk caches does not make sense: There
> is no cache to store a, say, 26MB object. If Squid lacks the
> corresponding configuration "lint" check, somebody should add it.
>
>
>> This config is giving 4% of cache gain ratio, 
>
>> in addition as i already mentionned before if i take the same above
>> config without worker and cach_dir with the same traffiic using aufs on
>> one of the disks ,  i have automatically i har 60 % cache ratio.
>
> When using AUFS, do you limit disk-cached object sizes to 32KB like you
> do with rock? If not, then you should remove the max-size limit from
> rock cache_dirs. Modern rock cache_dirs are capable of storing large
> objects.
>
> What kind of hit ratio do you get with rock if you do not limit
> swap-rate and do not specify swap-timeout?
>
> What kind of hit ratio do you get with rock if you use one worker, one
> rock cache_dir, do not limit swap-rate, do not specify swap-timeout, and
> start Squid with -N to disable SMP?
>
> As you can see, I am trying to understand whether the size limitation,
> the rate limiting, or SMP problems explain the drop in hit ratio.
>
>
>> Shoud rock give me the same performance as aufs ?
>
> It is a difficult question to answer correctly (for me). The goal is for
> rock performance to exceed that of (a)ufs, but I doubt we have reached
> that goal in every environment that matters (including yours).
>
> * In a non-SMP environment, I would expect similar hit ratios in most
> cases, but I would not be surprised if there are significant exceptions.
> Rock is focused on SMP support, and there are complexities/costs
> associated with SMP. Rock is getting better, but there are some known
> areas where rock cannot yet do what ufs (including aufs) can.
>
> * In a SMP environment, the question is mostly meaningless because there
> is no SMP support for ufs-based caches. Folks use squid.conf
> preprocessor hacks to configure ufs-based caches in SMP mode, but those
> setups usually violate HTTP and may cause serious problems. YMMV.
>
>
>> for a traffic of 1 Gb/s , is there a way to use aufs ?
>
> Before trying unsupported combinations of aufs and SMP, I would try to
> understand why your hit ratio is so low with rock. The questions above
> may be a good start in that investigation.
>
>
> Cheers,
>
> Alex.
>
>
>> ------------------------------------------------------------------------
>> *From:* Alex Rousskov <rousskov@xxxxxxxxxxxxxxxxxxxxxxx>
>> *Sent:* Thursday, July 2, 2020 4:24 PM
>> *To:* patrick mkhael <patrick.mkhael@xxxxxxxxxxx>;
>> squid-users@xxxxxxxxxxxxxxxxxxxxx <squid-users@xxxxxxxxxxxxxxxxxxxxx>
>> *Subject:* Re: [squid-users] rock issue
>>  
>> On 7/1/20 4:45 PM, patrick mkhael wrote:
>>
>>> ***Please note that you have 20 kids worth mapping (10 workers and 10
>>> diskers), but you map only the first 10.​{since i did not get the point
>>> of the diskers ,as far as i understood  , it should be like  (simple
>>> example)
>>
>>>> workers 2
>>>> cpu_affinity_map process_numbers=1,2,3,4 cores=1,2,3,4
>>>> cache_dir rock ...
>>>> cache_dir rock ...
>>
>> The above looks OK. Each worker is a kid process. Each rock cache_dir is
>> a kid process (we call them diskers).  If you have physical CPU cores to
>> spare, give each kid process its own physical core. Otherwise, give each
>> worker process its own physical core (if you can). Diskers can share
>> physical cores with less harm because they usually do not consume much
>> CPU cycles. Squid wiki has more detailed information about that:
>> https://wiki.squid-cache.org/Features/SmpScale#How_to_configure_SMP_Squid_for_top_performance.3F
>>
>>
>>> ***Why do you have 10 rock caches of various sizes? [ to be honest , i
>>> saw in many websites that it should be like this from the smallest to
>>> the bigest with diff size, i tought it should serve from small size pool
>>> to high ]
>>
>> IMHO, you should stop reading those web pages :-). There is no general
>> need to segregate objects by sizes, especially when you are using the
>> same slot size for all cache_dirs. Such segregation may be necessary in
>> some special cases, but we have not yet established that your case is
>> special.
>>
>>
>>> *****How many independent disk spindles (or equivalent) do you have? [ i
>>> have one raid 5 ssd disks , used by the 10 rock cache dir]
>>
>> Do not use RAID. If possible, use one rock cache_dir per SSD disk. The
>> only reason this may not be possible, AFAICT, is if you want to cache
>> more (per SSD disk) than a single Squid cache_dir can hold, but I would
>> not worry about overcoming that limit at the beginning. If you want to
>> know more about the limit, look for "33554431" in
>> http://www.squid-cache.org/mail-archive/squid-users/201312/0034.html
>>
>>
>>> ***How did you select the swap rate limits and timeouts for
>>> cache_dirs?[I took it also from online forum , can i leave it empty for
>>> both]
>>
>> If you want a simple answer, then it is "yes". Unfortunately, there is
>> no simple correct answer to that question. To understand what is going
>> on and how to tune things, I recommend studying the Performance Tuning
>> section of https://wiki.squid-cache.org/Features/RockStore
>>
>>
>>> ****Do you see any ERRORs or WARNINGs in cache log?[NO error or warning
>>> found in cache]
>>
>> Good. I assume you do see some regular messages in cache.log. Keep an
>> eye for ERRORs and WARNINGs as you change settings.
>>
>>
>> HTH,
>>
>> Alex.
>

_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users

[Index of Archives]     [Linux Audio Users]     [Samba]     [Big List of Linux Books]     [Linux USB]     [Yosemite News]

  Powered by Linux