On 08/31/2018 01:06 PM, Reiner Keller wrote: > Hello, > > Am 31.08.2018 um 13:59 schrieb Shyam Ranganathan: >> I suspect you have hit this: >> https://bugzilla.redhat.com/show_bug.cgi?id=1602262#c5 >> >> I further suspect your older setup was 3.10 based and not 3.12 based. >> >> There is an additional feature added in 3.12 that stores GFID to path >> conversion details using xattrs (see "GFID to path" in >> https://docs.gluster.org/en/latest/release-notes/3.12.0/#major-changes-and-features >> ) >> >> Due to which xattr storage limit is reached/breached on ext4 based bricks. >> >> To check if you are facing similar issue to the one in the bug provided >> above, I would check if the brick logs throw up the no space error on a >> gfid2path set failure. > > thanks for the hint. > > From log output (= no gfid2path errors) it seems to be not the problem > although the old > gluster volume was setup with version 3.10.x (or even 3.8.x i think). > > I wrote I could reproduce it on new ext4 and on old xfs gluster volumes > with version > 3.12.13 while it was running fine with ~ 3.12.8 (half year ago) without > problems. > > But just saw that my old main volume wasn't/isn't xfs but also ext4. > Digging into logs I could see that I was running in January still 3.10.8 > / 3.10.9 > and initial switched in April to 3.12.9 / 3.12 version branch. > > From entry sizes/differences your suggestion would fit: > > https://manpages.debian.org/testing/manpages/xattr.7.en.html or > http://man7.org/linux/man-pages/man5/attr.5.html > > In the current ext2, ext3, and ext4 filesystem implementations, the > total bytes used by the names and values of all of a file's extended > attributes must fit in a single filesystem block (1024, 2048 or 4096 > bytes, depending on the block size specified when the filesystem was > created). > > because I can see differences by volume setup type: <huge snip> So in short, the inode size limits in ext4 impacts the hard link counts that can be created in Gluster, which is the limitation that you hit, would that be a correct summary? > > >> To check if you are facing similar issue to the one in the bug provided >> above, I would check if the brick logs throw up the no space error on a >> gfid2path set failure. > > Is there some parameter to get more detailed error logging ? But from > docu it looks like it has default good settings: The error logs posted are from the client (FUSE mount) logs, the log lines with the gfid2path that I was mentioning is on the bricks. There is no further logging level that needs to change to see the said errors as these are warning and above. >> >>> My search for documentation found only the parameter >>> "storage.max-hardlinks" with default of 100 for version 4.0. >>> I checked it in my gluster 3.12.13 but here the parameter is not yet >>> implemented. > > If this problem is backend filesystem related it would be good to have > it documented also for 4.0 that the storage.max-hardlinks parameter > would work only if the backend is e.g. xfs and has enough inode space > for it (best with a reference/short example howto calculate it) ? Fair point, raised a github issue around the same here [1] (contributions welcome :) ). Regards, Shyam [1] Gluster documentation github issue for hardlink and ext4 limitations: https://github.com/gluster/glusterdocs/issues/418 _______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx https://lists.gluster.org/mailman/listinfo/gluster-users
