|
dear alex,
Was
this 7% measured with max-size=32000 or without?
[ i did not use max-size option]
When
using AUFS (without rock), do you limit disk-cached object sizes to
~32KB (max-size=32000)?
[
i use maximum_object_size_in_memory
250 MB and maximum_object_size 2 GB] // which i also use it in rock ]
squid
version [ Version 4.8]
thank u
From: Alex Rousskov <rousskov@xxxxxxxxxxxxxxxxxxxxxxx>
Sent: Tuesday, July 7, 2020 4:32 PM To: patrick mkhael <patrick.mkhael@xxxxxxxxxxx>; squid-users@xxxxxxxxxxxxxxxxxxxxx <squid-users@xxxxxxxxxxxxxxxxxxxxx> Subject: Re: [squid-users] rock issue On 7/7/20 6:26 AM, patrick mkhael wrote:
> **What kind of hit ratio do you get with rock if you do not > limit swap-rate and do not specify swap-timeout? [i also removed the max > size as recomended], the gain ratio is max 13 %. Noted, thank you. > **What kind of hit ratio do you get with rock if you use one worker, one > rock cache_dir, do not limit swap-rate, do not specify swap-timeout, and > start Squid with -N to disable SMP? [ as recomended, only one rock > cache_dir , no limit swap and excuted with -N option,the gain ration is 7%] When using AUFS (without rock), do you limit disk-cached object sizes to ~32KB (max-size=32000)? Finally, what is your Squid version? Thank you, Alex. > ------------------------------------------------------------------------ > *From:* Alex Rousskov <rousskov@xxxxxxxxxxxxxxxxxxxxxxx> > *Sent:* Saturday, July 4, 2020 3:40 AM > *To:* patrick mkhael <patrick.mkhael@xxxxxxxxxxx>; > squid-users@xxxxxxxxxxxxxxxxxxxxx <squid-users@xxxxxxxxxxxxxxxxxxxxx> > *Subject:* Re: [squid-users] rock issue > > On 7/3/20 4:50 AM, patrick mkhael wrote: > >> workers 3 >> cpu_affinity_map process_numbers=1,2,3,4,5,6 cores=1,2,3,4,5,6 >> cache_dir rock /rock1 200000 max-size=32000 swap-timeout=300 max-swap-rate=100 >> cache_dir rock /rock2 200000 max-size=32000 max-swap-rate=100 swap-timeout=300 >> cache_dir rock /rock3 200000 max-size=32000 max-swap-rate=100 swap-timeout=300 >> cache_mem 17 GB >> maximum_object_size_in_memory 25 MB >> maximum_object_size 1 GB >> cache_miss_revalidate off >> quick_abort_pct 95 > > > FYI: The combination of 1GB maximum_object_size and much smaller size > limits for objects in memory and disk caches does not make sense: There > is no cache to store a, say, 26MB object. If Squid lacks the > corresponding configuration "lint" check, somebody should add it. > > >> This config is giving 4% of cache gain ratio, > >> in addition as i already mentionned before if i take the same above >> config without worker and cach_dir with the same traffiic using aufs on >> one of the disks , i have automatically i har 60 % cache ratio. > > When using AUFS, do you limit disk-cached object sizes to 32KB like you > do with rock? If not, then you should remove the max-size limit from > rock cache_dirs. Modern rock cache_dirs are capable of storing large > objects. > > What kind of hit ratio do you get with rock if you do not limit > swap-rate and do not specify swap-timeout? > > What kind of hit ratio do you get with rock if you use one worker, one > rock cache_dir, do not limit swap-rate, do not specify swap-timeout, and > start Squid with -N to disable SMP? > > As you can see, I am trying to understand whether the size limitation, > the rate limiting, or SMP problems explain the drop in hit ratio. > > >> Shoud rock give me the same performance as aufs ? > > It is a difficult question to answer correctly (for me). The goal is for > rock performance to exceed that of (a)ufs, but I doubt we have reached > that goal in every environment that matters (including yours). > > * In a non-SMP environment, I would expect similar hit ratios in most > cases, but I would not be surprised if there are significant exceptions. > Rock is focused on SMP support, and there are complexities/costs > associated with SMP. Rock is getting better, but there are some known > areas where rock cannot yet do what ufs (including aufs) can. > > * In a SMP environment, the question is mostly meaningless because there > is no SMP support for ufs-based caches. Folks use squid.conf > preprocessor hacks to configure ufs-based caches in SMP mode, but those > setups usually violate HTTP and may cause serious problems. YMMV. > > >> for a traffic of 1 Gb/s , is there a way to use aufs ? > > Before trying unsupported combinations of aufs and SMP, I would try to > understand why your hit ratio is so low with rock. The questions above > may be a good start in that investigation. > > > Cheers, > > Alex. > > >> ------------------------------------------------------------------------ >> *From:* Alex Rousskov <rousskov@xxxxxxxxxxxxxxxxxxxxxxx> >> *Sent:* Thursday, July 2, 2020 4:24 PM >> *To:* patrick mkhael <patrick.mkhael@xxxxxxxxxxx>; >> squid-users@xxxxxxxxxxxxxxxxxxxxx <squid-users@xxxxxxxxxxxxxxxxxxxxx> >> *Subject:* Re: [squid-users] rock issue >> >> On 7/1/20 4:45 PM, patrick mkhael wrote: >> >>> ***Please note that you have 20 kids worth mapping (10 workers and 10 >>> diskers), but you map only the first 10.{since i did not get the point >>> of the diskers ,as far as i understood , it should be like (simple >>> example) >> >>>> workers 2 >>>> cpu_affinity_map process_numbers=1,2,3,4 cores=1,2,3,4 >>>> cache_dir rock ... >>>> cache_dir rock ... >> >> The above looks OK. Each worker is a kid process. Each rock cache_dir is >> a kid process (we call them diskers). If you have physical CPU cores to >> spare, give each kid process its own physical core. Otherwise, give each >> worker process its own physical core (if you can). Diskers can share >> physical cores with less harm because they usually do not consume much >> CPU cycles. Squid wiki has more detailed information about that: >> https://wiki.squid-cache.org/Features/SmpScale#How_to_configure_SMP_Squid_for_top_performance.3F >> >> >>> ***Why do you have 10 rock caches of various sizes? [ to be honest , i >>> saw in many websites that it should be like this from the smallest to >>> the bigest with diff size, i tought it should serve from small size pool >>> to high ] >> >> IMHO, you should stop reading those web pages :-). There is no general >> need to segregate objects by sizes, especially when you are using the >> same slot size for all cache_dirs. Such segregation may be necessary in >> some special cases, but we have not yet established that your case is >> special. >> >> >>> *****How many independent disk spindles (or equivalent) do you have? [ i >>> have one raid 5 ssd disks , used by the 10 rock cache dir] >> >> Do not use RAID. If possible, use one rock cache_dir per SSD disk. The >> only reason this may not be possible, AFAICT, is if you want to cache >> more (per SSD disk) than a single Squid cache_dir can hold, but I would >> not worry about overcoming that limit at the beginning. If you want to >> know more about the limit, look for "33554431" in >> http://www.squid-cache.org/mail-archive/squid-users/201312/0034.html >> >> >>> ***How did you select the swap rate limits and timeouts for >>> cache_dirs?[I took it also from online forum , can i leave it empty for >>> both] >> >> If you want a simple answer, then it is "yes". Unfortunately, there is >> no simple correct answer to that question. To understand what is going >> on and how to tune things, I recommend studying the Performance Tuning >> section of https://wiki.squid-cache.org/Features/RockStore >> >> >>> ****Do you see any ERRORs or WARNINGs in cache log?[NO error or warning >>> found in cache] >> >> Good. I assume you do see some regular messages in cache.log. Keep an >> eye for ERRORs and WARNINGs as you change settings. >> >> >> HTH, >> >> Alex. > |
_______________________________________________ squid-users mailing list squid-users@xxxxxxxxxxxxxxxxxxxxx http://lists.squid-cache.org/listinfo/squid-users
