I did one upgrade only from 3.7.9 to 3.7.10 and it is not a rolling upgrade, I stopped volume and then upgrade all the components. I will try restarting the volume and see if it helps.. On Mon, Apr 18, 2016 at 10:17 AM, Ashish Pandey <aspandey@xxxxxxxxxx> wrote: > Hi Serkan, > > I have gone through the logs and can see there are some blocked inode lock > requests. > We have observed that some other user have also faced this issue with > similar logs. > I think you have tried some rolling update on your setup or some NODES , on > which you have collected these statedumps, must have gone down for one or > other reason. > > We will further dig it up and will try to find out the root cause. Till than > you can resolve this issue by restarting the volume which will restart nfs > and shd and will release any locks taken by these process. > > "gluster volume start <volname> force" will do the same. > > Regards, > Ashish > > > ________________________________ > From: "Serkan Çoban" <cobanserkan@xxxxxxxxx> > To: "Ashish Pandey" <aspandey@xxxxxxxxxx> > Cc: "Gluster Users" <gluster-users@xxxxxxxxxxx>, "Gluster Devel" > <gluster-devel@xxxxxxxxxxx> > Sent: Monday, April 18, 2016 11:51:37 AM > > Subject: Re: Assertion failed: ec_get_inode_size > > You can find the statedumps of server and client in below link. > Gluster version is 3.7.10, 78x(16+4) disperse setup. 60 nodes named > node185..node244 > https://www.dropbox.com/s/cc2dgsxwuk48mba/gluster_statedumps.zip?dl=0 > > > On Fri, Apr 15, 2016 at 9:52 PM, Ashish Pandey <aspandey@xxxxxxxxxx> wrote: >> >> Actually it was my mistake I overlooked the configuration you provided..It >> will be huge. >> I would suggest to take statedump on all the nodes and try to grep for >> "BLOCKED" in statedump files on all the nodes. >> See if you can see any such line in any file and send those files. No need >> to send statedump of all the bricks.. >> >> >> >> >> ________________________________ >> From: "Serkan Çoban" <cobanserkan@xxxxxxxxx> >> To: "Ashish Pandey" <aspandey@xxxxxxxxxx> >> Cc: "Gluster Users" <gluster-users@xxxxxxxxxxx>, "Gluster Devel" >> <gluster-devel@xxxxxxxxxxx> >> Sent: Friday, April 15, 2016 6:07:00 PM >> >> Subject: Re: Assertion failed: ec_get_inode_size >> >> Hi Asish, >> >> Sorry for the question but do you want all brick statedumps from all >> servers or all brick dumps from one server? >> All server brick dumps is nearly 700MB zipped.. >> >> On Fri, Apr 15, 2016 at 2:16 PM, Ashish Pandey <aspandey@xxxxxxxxxx> >> wrote: >>> >>> To get the state dump of fuse client- >>> 1 - get the PID of fuse mount process >>> 2 - kill -USR1 <PID> >>> >>> statedump can be found in the same directory where u get for brick >>> process. >>> >>> Following link could be helpful for future reference - >>> >>> >>> https://github.com/gluster/glusterfs/blob/master/doc/debugging/statedump.md >>> >>> Ashish >>> >>> ________________________________ >>> From: "Serkan Çoban" <cobanserkan@xxxxxxxxx> >>> To: "Ashish Pandey" <aspandey@xxxxxxxxxx> >>> Cc: "Gluster Users" <gluster-users@xxxxxxxxxxx>, "Gluster Devel" >>> <gluster-devel@xxxxxxxxxxx> >>> Sent: Friday, April 15, 2016 4:02:20 PM >>> Subject: Re: Assertion failed: ec_get_inode_size >>> >>> Yes it is only one brick which error appears. I can send all other >>> brick dumps too.. >>> How can I get state dump in fuse client? There is no gluster command >>> there.. >>> _______________________________________________ >>> Gluster-users mailing list >>> Gluster-users@xxxxxxxxxxx >>> http://www.gluster.org/mailman/listinfo/gluster-users >>> >> _______________________________________________ >> Gluster-users mailing list >> Gluster-users@xxxxxxxxxxx >> http://www.gluster.org/mailman/listinfo/gluster-users >> > _______________________________________________ > Gluster-users mailing list > Gluster-users@xxxxxxxxxxx > http://www.gluster.org/mailman/listinfo/gluster-users > _______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-users
