Search squid archive

Re: Re: does rock type deny being dedicated to specific process ??

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 29/10/2013 10:28 p.m., Dr.x wrote:

hi amos ,
===============================================
Question 1 :

about aufs dir ,
  about smp ,
you say that it dont work with smp
but my question is more accurate ,
i c that it dont share workers ,  but it works if we assign  specific
process to specific cache dir .

in my opinion we could solve our problem of cpu load partially ??!!!
because without smp , all cache dir aufs or not aufs , were sharing the same
process , and this process was not getting beneift of all cores ""uptill now
this is my understanding "

again , my question is ,
we should not  say that auf dir dont work with  smp  completely

Yes we should.

SMP means "symmetric multi-processing". It is achieved in Squid by sharing memory blocks between workers, and inter-process messages about processing state.
 AUFS memory index contents and changes are *not* shared between workers.
=> Therefore "AUFS is not SMP-enabled" ... as we keep saying.

Also, if you attempt to use one AUFS cache_dir for two workers, each worker handles the AUFS disk cache as if it were the *only* process accessing it:

a) Worker 1 will save or update content on disk without informing worker 2. Causing a random URL in worker 2 to point at the on-disk object saved by worker 1 from another URL. This also happens the other way around as worker 2 over-write objects of worker 1.

b) Worker 1 and 2 will occasionally save different data to the same disk file at the same time. Causing disk file corruption and also leading to problem (a).

c) URL content stored by worker 1 will not be notified to worker 2, so it will never HIT on that content until it has downloaded a copy of its own. Which will be stored in cache, either resulting in problem (a) immediately or causing worker 2 to add duplicate object in the cache until such time as problem (a) causes it to get corrupted by worker 1.

Squid contains some protection against disk corruption like this. What you will see is that when Squid starts up you get some HITs from that cache by each worker. As the cache contents ages and gets revalidated the HITs slowly all become SWAPFAIL_MISS and most of your existing disk contents get erased.


, but to be
accurate we can  say  that aufs dont work with shared workers

You can only say this because workers and SMP are different things.
Worker is the type of process instance.
SMP is a design mechanism for how workers might interoperate.

AUFS is multi-threaded. Which is very similar to SMP but without the P.

, but we can
get benefit of smp to load cpu cores and set each core to instance aufs dir
and as a result we solved our cpu load partially ,

The "A" stands for Asynchronous and is an algorithm for scalable multi-threaded I/O. The OS kernel decides which core each thread runs on.

AUFS is designed currently to benefit from multiple cores with only single-process. This does not change despite how many workers you use.


i just want to understand this issue and with to correct me about the info i
posted here ,

Complexity.

Each component in Squid has the three different concept properties (multi-process Y/N, multi-threaded Y/N, shared-memory Y/N).
AUFS in Squid right now is the particular combination of (No, Yes, No).

Amos


===============================================

Amos Jeffries-2 wrote
PS:  if you want to experiment, you could try given the frontend and
backend config two slightly different cache_dir lines. So the frontend
has a "read-only" flag but otherwise identical settings. In theory that
would make the frontend able to HIT on the rock cache, but only the
backends able to store things there.

Amos
not understanding yet ,
why add to front end a cahe dir ???

So it can respond with HIT immediately.
Each request passed to the backend needs serializing into HTTP text protocol, sending to the backend, then re-parsing by the backend. The reply coming back also has to go through the same number of extra parse+serialize steps on its way back. This is slower than a shared-memory lookup and sending the HIT back directly.

Note: The main benefit of CARP is to de-duplicate cached contents while simultaneously sharing traffic load between processes. These same properties are gained by workers in parallel sharing a rock cache. The remaining problem in Squid is that the rock cache max-size limit of 32KB. Once that is gone (expected in Squid-3.5) there should not be any need for CARP frontend/backend separation, all workers can be caching and frontend.

did u mean to put cache dir into  process squid # 1??

I mean have process #1 have read-only access to the rock cache. So it can benefit from quick HIT, but not cause problems with both frontend and backend storing same transaction to different places in the cache.

Amos




[Index of Archives]     [Linux Audio Users]     [Samba]     [Big List of Linux Books]     [Linux USB]     [Yosemite News]

  Powered by Linux