Re: Gluster status fails

[Atin Mukherjee <amukherj@xxxxxxxxxx>]

On Wed, 30 Aug 2017 at 20:55, mohammad kashif <kashif.alig@xxxxxxxxx> wrote:

Hi

I am running a 400TB  five node purely distributed gluster setup. I am troubleshooting an issue where some times files creation fails. I found that volume status is not working

gluster volume status
Another transaction is in progress for atlasglust. Please try again after sometime.

When I tried from other node then it seems two nodes have Locking issue

gluster volume status
Locking failed on pplxgluster01... Please check log file for details.
Locking failed on pplxgluster04... Please check log file for details.

This suggests that there are concurrent gluster cli operations been performed on the same volume. Are you monitoring the cluster through nagios or you have a script on all the nodes which checks for volume's health in a period of interval? Please note glusterd will process one cli operation on a volume at one time, the rest all transactions on the same volume will be failed.

Also noticed that glusterfsd process is using around 1000% cpu usage.  It is a decent server with 16 core and 64GB RAM.

Gluster version is 3.11.2-1

Can you please suggest that how to troubleshoot further?

Thanks

Kashif

 

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://lists.gluster.org/mailman/listinfo/gluster-users

--

- Atin (atinm)

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://lists.gluster.org/mailman/listinfo/gluster-users