On 8/28/06, Adrian Chadd <adrian@xxxxxxxxxxxxxxx> wrote:
On Mon, Aug 28, 2006, Pranav Desai wrote: > for (a), shouldnt the disk queue also affect the performance during > the first phase of the test. The first phase is almost perfect with no > degradation in times. > If I may send you the graphs, in which it seems very wrong that the > degradation happens so suddenly. > Maybe I will try a few test with a longer first phase to see if the > disk queue catches up and degrades the performance.
Ok .. this time I tried a 2 phase x 4hr test with a very small fill phase. But with a tmpfs, so as to completely avoid any issues with disk. The cache store was 6GB. It still showed the same weirdness. The second phase completely bombed out. I am pretty sure that if I run the test with the same setup without the idle phase i.e. one long phase for 8 hr, it will do just fine. So there is something that it is doing during the idle phase, maybe trying to cleanup the queues or something that its just not able to recover. And when the phase 2 start (which is like 15min after the idle phase) its not able to sustain any serious load. Some other observations from mrtg graphs (http://pd.dnsalias.org/squid/Cache.html) - The values after midnight is the test. - The CPU utilization climbs up. - number of reads/s (cacheSysPerf.cacheSysNumReads) drops drastically, from 1000/sec in first phase to about 200/s in second phase. Basically, the sustained req. rate during the second phase is ~ 200req/s v/s a 1000 req/s in first phase. And this is not a gradually decrease. I will try the same test with 2.5, to see if I see the same results.
I'm pretty sure the first phase is "fill" and not "read". > I will be trying COSS shortly. Even I am expecting it to do much > better. What other kind of work do you have in mind. Another > filesystem or something ? Yes, another filestore. And a bunch of squid storage manager changes to boot. > I am thinking of giving this http://logfs.sourceforge.net/ a try. > But I dont know if it needs changes in squid to make efficient use of > the filesystem. There's lots which could be done to improve Squid's use of traditional file stores. In fact, there's bunches of papers written from circa 8 years ago when academics were really interested in this stuff. A few of them twiddled with the Squid store code and made things faster but didn't contribute any patches back. The only person who did contribute major store code changes (Eric Stern) contributed his COSS code which we've only made stable (and useable under modern loads) very recently. So yes. There's plenty of room to grow. I'm kind of hoping someone else will pop up out of the woodwork and help us work on this. Please :) Adrian
-- ------------------------------ http://pd.dnsalias.org
