Hi, I think I know what happened. According to the logs, the crawlers recieved a signum(15). They seemed to have died before having finished. Probably too much to do simultaneously. I have disabled and re-enabled quota and will set the quotas again with more time. Is there a way to restart a crawler that was killed too soon? If I restart a server while a crawler is running, will the crawler be restarted, too? We'll need to do some hardware fixing on one of the servers soon and I need to know whether I have to check the crawlers first before shutting it down. Thanks for the pointers Gudrun Amedick Am Dienstag, den 20.11.2018, 11:38 +0530 schrieb Hari Gowtham: > Hi, > > Can you check if the quota crawl finished? Without it having finished > the quota list will show incorrect values. > Looking at the under accounting, it looks like the crawl is not yet > finished ( it does take a lot of time as it has to crawl the whole > filesystem). > > If the crawl has finished and the usage is still showing wrong values > then there should be an accounting issue. > The easy way to fix this is to try restarting quota. This will not > cause any problems. The only downside is the limits won't hold true > while the quota is disabled, > till its enabled and the crawl finishes. > Or you can try using the quota fsck script > https://review.gluster.org/#/c/glusterfs/+/19179/ to fix your > accounting issue. > > Regards, > Hari. > On Mon, Nov 19, 2018 at 10:05 PM Frank Ruehlemann > <f.ruehlemann@xxxxxxxxxxxxxx> wrote: > > > > > > Hi, > > > > we're running a Distributed Dispersed volume with Gluster 3.12.14 at > > Debian 9.6 (Stretch). > > > > We migrated our data (>300TB) from a pure Distributed volume into this > > Dispersed volume with cp, followed by multiple rsyncs. > > After the migration was successful we enabled quotas again with "gluster > > volume quota $VOLUME enable", which finished successfully. > > And we set our required quotas with "gluster volume quota $VOLUME > > limit-usage $PATH $QUOTA", which finished without errors too. > > > > But our "gluster volume quota $VOLUME list" shows wrong values. > > For example: > > A directory with ~170TB of data shows only 40.8TB Used. > > When we sum up all quoted directories we're way under the ~310TB that > > "df -h /$volume" shows. > > And "df -h /$volume/$directory" shows wrong values for nearly all > > directories. > > > > All 72 8TB-bricks and all quota deamons of the 6 servers are visible and > > online in "gluster volume status $VOLUME". > > > > > > In quotad.log I found multiple warnings like this: > > > > > > [2018-11-16 09:21:25.738901] W [dict.c:636:dict_unref] (-->/usr/lib/x86_64-linux-gnu/glusterfs/3.12.14/xlator/features/quotad.so(+0x1d58) > > > [0x7f6844be7d58] -->/usr/lib/x86_64-linux-gnu/glusterfs/3.12.14/xlator/features/quotad.so(+0x2b92) [0x7f6844be8b92] -->/usr/lib/x86_64-linux- > > > gnu/libglusterfs.so.0(dict_unref+0xc0) [0x7f684b0db640] ) 0-dict: dict is NULL [Invalid argument] > > In some brick logs I found those: > > > > > > [2018-11-19 07:23:30.932327] I [MSGID: 120020] [quota.c:2198:quota_unlink_cbk] 0-$VOLUME-quota: quota context not set inode (gfid:f100f7a9-0779- > > > 4b4c-880f-c8b3b4bdc49d) [Invalid argument] > > and (replaced the volume name with "$VOLUME") those: > > > > > > The message "W [MSGID: 120003] [quota.c:821:quota_build_ancestry_cbk] 0-$VOLUME-quota: parent is NULL [Invalid argument]" repeated 13 times > > > between [2018-11-19 15:28:54.089404] and [2018-11-19 15:30:12.792175] > > > [2018-11-19 15:31:34.559348] W [MSGID: 120003] [quota.c:821:quota_build_ancestry_cbk] 0-$VOLUME-quota: parent is NULL [Invalid argument] > > I already found that setting the flag "trusted.glusterfs.quota.dirty" might help, but I'm unsure about the consequences that will be triggered. > > And I'm unsure about the necessary version flag. > > > > Has anyone an idea how to fix this? > > > > Best Regards, > > -- > > Frank Rühlemann > > IT-Systemtechnik > > > > UNIVERSITÄT ZU LÜBECK > > IT-Service-Center > > > > Ratzeburger Allee 160 > > 23562 Lübeck > > Tel +49 451 3101 2034 > > Fax +49 451 3101 2004 > > ruehlemann@xxxxxxxxxxxxxxxxxxx > > www.itsc.uni-luebeck.de > > > > > > > > > > _______________________________________________ > > Gluster-users mailing list > > Gluster-users@xxxxxxxxxxx > > https://lists.gluster.org/mailman/listinfo/gluster-users > >
Attachment:
smime.p7s
Description: S/MIME cryptographic signature
_______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx https://lists.gluster.org/mailman/listinfo/gluster-users
