On Fri, Aug 7, 2015 at 1:36 AM, Bob Ababurko <bob@xxxxxxxxxxxx> wrote: > @John, > > Can you clarify which values would suggest that my metadata pool is too > slow? I have added a link that includes values for the "op_active" & > "handle_client_request"....gathered in a crude fashion but should hopefully > give enough data to paint a picture of what is happening. > > http://pastebin.com/5zAG8VXT Dividing by the first 20s of the second sample period, you're seeing ~750 client metadata operations handled per second, which is kind of a baseline level of performance (a little better than what I get running a ceph cluster locally on my workstation). That's probably corresponding to roughly the same number of file creates per second -- your workload is very much a small file one, where "files per second" is a much more meaningful measure than IOPS or MB/s. It does look like the kind of pattern where you've got a large clutch of several thousand metadata pool rados ops coming out every few seconds, then draining out over a few seconds. Your metadata pool isn't pathologically slow (it's completing at least hundreds of ops per second), but it is noticeable that during some periods where op_active is draining, handle_client_request is not incrementing -- i.e. client metadata ops are stalling while the MDS waits for its RADOS operations to complete. I can't say a massive amount beyond that, other than what you'd already figured out -- it would be worth trying to put some faster storage in for your metadata pool. John _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |