Hi Dan - I've seen this problem too. I agree with everything you've described - seems to happen more quickly on more heavily used volumes, and a restart fixes it right away. I've also been considering writing a cronjob to fix this - have you made any progress on this, anything to report? I'm running a fairly simple distributed, non-replicated volume across two servers. What sort of tasks are you using your gluster for? Ours is for a render farm, so we see a very large number of mounts/unmounts as render nodes mount various parts of the filesystem. I wonder if this has anything to do with it; is your use case anything similar? - brian On 4/17/12 7:30 PM, Dan Bretherton wrote: > Dear All- > I find that I have to restart glusterd every few days on my servers to > stop NFS performance from becoming unbearably slow. When the problem > occurs, volumes can take several minutes to mount and there are long > delays responding to "ls". Mounting from a different server, i.e. > one not normally used for NFS export, results in normal NFS access > speeds. This doesn't seem to have anything to do with load because it > happens whether or not there is anything running on the compute > servers. Even when the system is mostly idle there are often a lot of > glusterfsd processes running, and on several of the servers I looked > at this evening there is a process called glusterfs using 100% of one > CPU. I can't find anything unusual in nfs.log or > etc-glusterfs-glusterd.vol.log on the servers affected. Restarting > glusterd seems to stop this strange behaviour and make NFS access run > smoothly again, but this usually only lasts for a day or two. > > This behaviour is not necessarily related to the length of time since > glusterd was started, but has more to do with the amount of work the > GlusterFS processes on each server have to do. I use a different > server to export each of my 8 different volumes, and the NFS > performance degradation seems to affect the most heavily used volumes > more than the others. I really need to find a solution to this > problem; all I can think of doing is setting up a cron job on each > server to restart glusterd every day, but I am worried about what side > effects that might have. I am using GlusterFS version 3.2.5. All > suggestions would be much appreciated. > > Regards, > Dan. > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
