Search squid archive

Re: Cache reference age for heap LRU/LFUDA and rock/aufs

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Fri, Feb 9, 2018 at 7:50 PM, Alex Rousskov <rousskov@xxxxxxxxxxxxxxxxxxxxxxx> wrote:

I cannot answer your question for aufs, but please note that rock
cache_dirs do not support/have/use a configurable replacement policy:
Each incoming object is assigned a slot based on its key hash. With
modern rock code, it is possible to remove that limitation IIRC, but
nobody have done that.

Yeah I figured this out from the source code and I'm extremely surprised by the fact that it was never mentioned in documentation. I think it will be a huge blocker in our squid 4 + SMP + rock migration plan.

So what does rock do when storage is full then?
 


> If you're wondering why would we need to know that – it's related to
> GDPR and removing data of closed customer's accounts. We need to make
> sure that we don't have any "not being accessed anymore" objects older
> than "data retention period" days.

If it is important to get this right, then I would not trust replacement
policy metadata with this: The corresponding code interfaces look
unreliable to me, and access counts/timestamps for a ufs-based cache_dir
are not updated across Squid restarts when the swap log is lost (at least).


It's actually fine, we never restart squid and if it restarted by any unexpected reason (host reboot, crash or w/e) we just replace the host.
 
I would instead configure Squid to prohibit serving hits that are too
old. That solution does not match your problem exactly, but it may be
good enough and should work a lot more reliably across all cache_dirs.
If there is no "age" ACL to use with the send_hit directive, then you
may need to add one.

    http://www.squid-cache.org/Doc/config/send_hit/

You may also be able to accomplish the same using refresh_pattern, but I
am a little worroed about various exceptional/special conditions
implemented on top of that directive. Others on this list may offer
better guidance in this area.


I was thinking about similar solution but this is exactly why I wasn't able to use it – there seems to be no acl suitable for such task.

We can always just replace the host every month or something like this but it'll mean starting with a cold cache every time which I wanted to avoid.

I found this debug option for heap which could probably help in understanding of approximate cache age but it doesn't work with rock because rock uses some "simple scan" policy.

> src/repl/heap/store_repl_heap.cc:        debugs(81, 3, "Heap age set to " << h->theHeap->age);


HTH,

Alex.



--
With best regards, Ivan Larionov.
_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users

[Index of Archives]     [Linux Audio Users]     [Samba]     [Big List of Linux Books]     [Linux USB]     [Yosemite News]

  Powered by Linux