Personally, I've seen Solaris bottlenecking on file opens in large directories. This was a while ago, but it was one of the major reason we switched to Linux -- the order of magnitude improvement in directory scale was sure handy for 80-90K users with no quota. The kind of blocking I'm talking about didn't show up in sar or iostat, despite being "IO" in nature. Of course it was some sort of algorithmic inefficiency in the directory data structures, not the speed of moving data on & off the disk. As a general statement, tho, finding bottlenecks like the one UCDavis is complaining about is done by process sampling. No guessing required. :wes On 06 Oct 2007, at 05:50, Rob Mueller wrote: >> The iostat and sar data disagrees with it being an I/O issue. >> >> 16 gigs of RAM with about 4-6 of it being used for Cyrus >> leaves plenty for ZFS caching. Our hardware seemed more than >> adequate to anyone we described it to. >> >> Yes beyond that it's anyone guess. > > If it wasn't IO limit related, and it wasn't CPU limit related, > then there must be some other single resource that things were > contending for. > > My only guess then is it's some global kernel lock or the like. > > When the load skyrocketed, it must have shown that lots of > processes were not in S (sleep) state. Were they in R (run) state > or D (io wait) state? Since you're on Solaris, you could use DTrace > to find out what they were actually doing/waiting for... ---- Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
