thanks. Before I try the gateway squid solution, I want to change one of my squid to use memory cache only. I have 16GB RAM. now cache_mem is set to 5GB. I will try to increase it to 12GB and set cache_dir to null schma. I do this because I am sure that my hot objects can be saved in RAM, non-hot objects created by robots will stale and the memory will be reused. Is that all I need to set squid to be a memory cache? 2011/8/18 Amos Jeffries <squid3@xxxxxxxxxxxxx>: > On 18/08/11 22:50, Chen Bangzhong wrote: >> >> thanks you Amos and Drunkard. >> >> My website hosts novels, That's, user can read novel there. >> >> The pages are not truely static contents, so I can only cache them for >> 10 minutes. >> >> My squids serve both non-cachable requests (works like nginx) and >> cachable-requests (10 min cache). So 60% cache miss is reasonable. It >> is not a good design, but we can't do more now. > > Oh well. Good luck wishes on that side of the problem. > >> >> Another point is, only hot novels are read by users. Crawlers/robots >> will push many objects to cache. These objects are rarely read by user >> and will expire after 10 minutes. >> >> If the http response header indicates it is not cachable(eg: >> max-age=0), will squid save the response in RAM or disk? My guess is >> squid will discard the response. > > Correct. It will discard the response AND anything it has already cached for > that URL. > > For non-hot objects this will not be a major problem. But may raise disk I/O > a bit as the existing old stored content gets kicked out. Which might > actually be a good thing, emptying space in the cache early. Or wasted I/O. > It's not clear exactly which. > >> >> If the http response header indicates it is cachable(eg: max-age=600), >> squid will save it in the cache_mem. If the object is larger than >> maximum_object_size_in_memory, it will be written to disk. > > Yes. > >> >> Can you tell me when will squid save the object to disk? When will >> squid delete the staled objects? > > Stale objects are deleted at the point they are detected as stale and no > longer usable (ie a request has been made for it and updated replacement has > arrived from the web server). Or if they are the oldest object stored and > more cache space is needed for newer objects. > > > Other than tuning your existing setup there are two things I think you may > be interested in. > > The first is a Measurement Factory project which involves altering Squid to > completely bypass the cache storage when an object can't be cached or > re-used by other clients. Makes them faster to process, and avoids dropping > cached objects to make room. Combining this with a "cache deny" rule > identifying those annoying robots as non-cacheable would allow you to store > only the real users traffic needs. > This is a slightly longer-term project, AFAIK it is not ready for > production use (might be wrong). At minimum TMF are possibly needing > sponsorship assistance to progress it faster. Contact Alex Rousskov about > possibilities there, http://www.measurement-factory.com/contact.html > > > The second thing is an alternative squid configuration which would emulate > that behaviour immediately using two Squid instances. > Basically; configure a new second instance as a non-caching gateway which > all requests go to first. That could pass the robots and other easily > detected non-cacheable requests straight to the web servers for service. > While passing the other potentially cacheable requests to your current Squid > instance, where storage and cache fetches happen more often without the > robots. > > The gateway squid would have a much smaller footprint since it needs no > memory for caching or indexing, and no disk usage at all. > > Amos > -- > Please be using > Current Stable Squid 2.7.STABLE9 or 3.1.14 > Beta testers wanted for 3.2.0.10 >
