On Wed, 23 Jun 2010 09:51:40 -0500, Seann Clark <nombrandue@xxxxxxxxxxxxxxx> wrote: > Amos Jeffries wrote: >> On Tue, 22 Jun 2010 16:05:04 -0500, Seann Clark >> <nombrandue@xxxxxxxxxxxxxxx> wrote: >> >>> All, >>> >>> I have been playing with/tweaking/breaking my squid for a few months >>> >> >> >>> now, and I am looking for suggestions from the list on improving >>> performance. This is on a home system, which does not have a large user >>> base. I am running a Dual Xeon 2.0 Ghz system with 2 gigs ram, 120 Gigs >>> hard drive, in a Raid 5 configuration controlled by a 3ware RAID card. >>> >> >> Optimization Tip #1: get rid of the RAID-5. >> http://wiki.squid-cache.org/SquidFaq/RAID >> >> > I am planning on adding in a new disk to the unit, to move the cache off > the RAID volume, and keep the RAID volume just for the Operating system. > Would that have any impact on squid if its binary lives on the RAID-5 > volume, but the caches are on separate physical drives? It would get faster. That is all. You can even do it live for no downtime: update squid.conf with new cache_dir paths run "squid -z" run "squid -k reconfigure" to shift the cache location in the running copy. remove old RAID directories. >>> I was using the stock Fedora 8 RPM for this, which was single threaded, >>> squid 2.6 Stable 22. I am also running this with diskd currently. >>> >> >> Optimization tip #2: run the fastest Squid release. >> Today 2.7 is current fastest official for production use. >> If you are willing to run semi-production code 2.HEAD daily snapshots >> have more speed experiments and some few high-performance testers behind >> it. >> >> >>> I have recently recompiled squid to the latest stable for version >>> 2.7 (Stable 9) with the async io flag passed to the configure command. >>> After a little updating of my configurations, just enough to get it to >>> work (I haven't changed any of the settings that are new to 2.7, so they >>> >> >> >>> are defaults right now), I have noticed a drastic improvement in speed >>> and even when the system is at a high load (3-5 on the system load, >>> >> >> So that covers #2. :) >> >> >> Optimization tip #3: use fastest disk IO method available. >> Fedora being a linux that would be AUFS. >> >> Specifically in 2.7, AUFS tuned via cache_dir directive for large >> objects >> (~1MB or more) alongside COSS directories for objects smaller than the >> AUFS >> holds. >> NP: these two caches types are happy to share one disk between them. An >> exception to the on-disk-per-cache_dir rule of thumb. >> >> >> > To get this a little clearer, as I am still learning what I can do with > the cache, I would configure the directives as such (example sizes): > > cache_dir coss /var/spool/coss_cache 1024 1 > cache_dir aufs /var/spool/bigcache 1024 100 There are three numbers needed (size in MB, number of top level directories, number of sub-directories) The options after those for AUFS you will need the min-size=N parameter and the COSS dir will need the max-size=N parameter (non-overlapping so one byte difference) http://www.squid-cache.org/Versions/v2/2.7/cfgman/cache_dir.html has full documentation on the directive and its settings. The specific relevant bits you want to read are under "The coss store type:" and "Common options:" at the bottom. > >>> reported by top) it runs fairly well. I am looking at moving my cache >>> directories off to a different disk, formatted with reiserFS, I am also >>> planning on running it using diskd as the storage scheme. The drive I am >>> >> >> >> see tip #3. AUFS makes use of multi-core threads, diskd process is >> single-threaded even if it runs additional to the Squid main thread. >> >> From my own experience, Squid has had no direct benefit from RieserFS >> over >> extN. The OS and other apps sometimes do gain, so it's up to you. >> >> You will need to tune the items which get journalled such as noatime etc >> to increase disk IO speeds either way. >> >> >> >>> using will be about 100 gigs of formatted space, and I plan to use all >>> that space for the cache. >>> >> >> Make sure you have enough RAM for the indexes. 2GB is borderline for your >> planned cache size. >> 4KB per object stored is what the index uses, with up to 2^31 objects per >> cache_dir = up to 8GB index. Reality varies with your users habits. >> >> NP: 2.7 in-memory handling of large objects is not great, so sacrificing >> any large cache_mem RAM cache for COSS small-object cache is probably a >> good trade-off in that release. >> >> > Expanded RAM in the system is planned, so it is a good point you > mentioned as it gives me a baseline to get for the system >>> I would like to know if this is a good plan, or should I change some >>> >> >> >>> things and how to change them, as well as any suggestions of >>> configuration settings with the cache, and 2.7 options that may make a >>> cache run even faster. >>> >>> >> >> On top of those basic parameters, your ACL configuration strongly >> influences speed. Read the wiki ACL pages on which are grouped as >> fast/slow >> (NP: 'fast' does not strictly mean speed there) to get the ordering >> optimal. >> Avoid any operations using regex like plague. >> Avoid helper operations as well since they often introduce large amounts >> of lag as well. >> >> Amos >> > I don't use regex in my configuration, and all my ACL's are set for > default access for snmp, cache access, and cachemanager access. > > ~Seann
