> Matus UHLAR - fantomas wrote: >> On 21.01.09 10:05, Andreev Nikita wrote: >> >>> Squid cache resides on NFS partition on storage system. It is 1Gb link >>> and both sides are connected to the same switch. Squid has dual core >>> Xeon processor and 4GB of RAM. >>> >>> The main concern here is that squid always eats 100% of 1 core. And >>> our clients can't reach full channel throughput (4Gbs). As I can see >>> outside link is half full. Secondly it looks like FS performance is >>> very poor. I tried to clear cache by setting >>> cache_swap_low 0 >>> cache_swap_high 0 >>> and it took about 15 hours for squid to actually clear the cache! >>> > > Yeah. Deleting a million objects over NFS is going to take a while. > >>> Why does squid eat 100% of processor if the problem is in FS? > > How is your cache_dir defined? aufs (in general) is a better choice > than ufs, diskd might still have some stability issues under load, and > coss is a good supplement as a small object cache. Conceivably if Squid > is set up with a ufs cache_dir mounted as NFS, it's spending a lot of > time in a wait state, blocked while the I/O completes. > >>> Maybe it's not an FS problem at all? > > I have to agree that using a cache_dir mounted as NFS is (at the very > least) part of the problem... > >>> What can I do to find the performance bottleneck? >>> > > Use profiling tools like iostat, vmstat and top. > >> >> NFS is the bottleneck. Can you connect the disk on storage system >> directly? >> > > Or at least use AoE* or iSCSI... NFS is convenient, but far from ideal > for a Squid cache. AoE and iSCSI are better**, but not neither will > perform as well as direct attached storage. The less latency between > Squid and the cache data, the better. > >> btw, which link is 1 gbit and which is 4 gbit? NFS is connected with >> 1gbit >> and you want squid to be able to saturate 4gbit connection? >> > > Can one Squid process even saturate a 4gbit link? In this case, given > the number of clients (~216) and the average number of requests per > minute (432.5) I'd have to guess we are talking about a 4mbit internet > connection. Yes it can. Squid's passing through of large objects is much more efficient than its pass-thru of small objects. A few dozen clients simultaneously grabbing movies or streaming through a CONNECT request can saturate a multi-GB link network buffer easily. A dual-core Xeon _should_ be able to saturate a 10GB link with all clients online. > > From the cache_info dump provided, DNS lookups are taking nearly 200 > ms. That's pretty slow too. Then again, I suppose if the Squid server > only has one Ethernet interface and that is saturated with NFS, DNS > queries are going to suffer. > > Chris > > *ATA over Ethernet (http://en.wikipedia.org/wiki/ATA_over_Ethernet) > **Just a gut feeling with no benchmarks to back it up, but I would think > that AoE would be better than iSCSI, as there is less protocol overhead. > Amos
