----- Original Message ----- > From: "Franco Broi" <franco.broi@xxxxxxxxxx> > To: "Pranith Kumar Karampuri" <pkarampu@xxxxxxxxxx> > Cc: gluster-users@xxxxxxxxxxx > Sent: Sunday, June 1, 2014 10:53:51 AM > Subject: Re: glusterfsd process spinning > > > The volume is almost completely idle now and the CPU for the brick > process has returned to normal. I've included the profile and I think it > shows the latency for the bad brick (data12) is unusually high, probably > indicating the filesystem is at fault after all?? I am not sure if we can believe the outputs now that you say the brick returned to normal. Next time it is acting up, do the same procedure and post the result. Pranith > > On Sun, 2014-06-01 at 01:01 -0400, Pranith Kumar Karampuri wrote: > > Franco, > > Could you do the following to get more information: > > > > "gluster volume profile <volname> start" > > > > Wait for some time, this will start gathering what operations are coming to > > all the bricks" > > Now execute "gluster volume profile <volname> info" > > > /file/you/should/reply/to/this/mail/with > > > > Then execute: > > gluster volume profile <volname> stop > > > > Lets see if this throws any light on the problem at hand > > > > Pranith > > ----- Original Message ----- > > > From: "Franco Broi" <franco.broi@xxxxxxxxxx> > > > To: gluster-users@xxxxxxxxxxx > > > Sent: Sunday, June 1, 2014 9:02:48 AM > > > Subject: glusterfsd process spinning > > > > > > Hi > > > > > > I've been suffering from continual problems with my gluster filesystem > > > slowing down due to what I thought was congestion on a single brick > > > being caused by a problem with the underlying filesystem running slow > > > but I've just noticed that the glusterfsd process for that particular > > > brick is running at 100%+, even when the filesystem is almost idle. > > > > > > I've done a couple of straces of the brick and another on the same > > > server, does the high number of futex errors give any clues as to what > > > might be wrong? > > > > > > % time seconds usecs/call calls errors syscall > > > ------ ----------- ----------- --------- --------- ---------------- > > > 45.58 0.027554 0 191665 20772 futex > > > 28.26 0.017084 0 137133 readv > > > 26.04 0.015743 0 66259 epoll_wait > > > 0.13 0.000077 3 23 writev > > > 0.00 0.000000 0 1 epoll_ctl > > > ------ ----------- ----------- --------- --------- ---------------- > > > 100.00 0.060458 395081 20772 total > > > > > > % time seconds usecs/call calls errors syscall > > > ------ ----------- ----------- --------- --------- ---------------- > > > 99.25 0.334020 133 2516 epoll_wait > > > 0.40 0.001347 0 4090 26 futex > > > 0.35 0.001192 0 5064 readv > > > 0.00 0.000000 0 20 writev > > > ------ ----------- ----------- --------- --------- ---------------- > > > 100.00 0.336559 11690 26 total > > > > > > > > > > > > Cheers, > > > > > > _______________________________________________ > > > Gluster-users mailing list > > > Gluster-users@xxxxxxxxxxx > > > http://supercolony.gluster.org/mailman/listinfo/gluster-users > > > > > _______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://supercolony.gluster.org/mailman/listinfo/gluster-users
