Coming back to the problem of quota sizes not converging, the converging may not complete if glusterd is restarted before converging process is complete because a thread is spawned by glusterd to do the converging. Hence, if glusterd is restarted converging will fail or to put it precisely it will not be completed. So running ls -lR on the mount point will solve the problem. On Mon, Sep 5, 2011 at 4:40 PM, Mohammed Junaid <junaid at gluster.com> wrote: > Hi Anthony, > > This is quiet unusual, to debug this further can you get the output of "du > -h" of mount-point and of directory soft (which is under mount point). This > is will help a lot in debugging this issue. Also, I noticed that the volume > info output provided is of a distributed-replicate setup which is different > from the one provided previously (it was plain replicate). Also please > provide the output of getfattr for the bricks and soft directory. > > Junaid > > On Mon, Sep 5, 2011 at 2:58 PM, anthony garnier <sokar6012 at hotmail.com>wrote: > >> >> Volume configuration: >> >> yval1000:/ # gluster volume info >> >> Volume Name: venus >> Type: Distributed-Replicate >> Status: Started >> Number of Bricks: 2 x 2 = 4 >> >> Transport-type: tcp >> Bricks: >> Brick1: ylal3020:/soft/venus >> Brick2: ylal3030:/soft/venus >> Brick3: yval1000:/soft/venus >> Brick4: yval1010:/soft/venus >> Options Reconfigured: >> features.quota-timeout: 120 >> features.limit-usage: /test:100MB,/psa:200MB,/:7GB,/soft:5GB >> features.quota: on >> geo-replication.indexing: on >> network.ping-timeout: 10 >> performance.cache-size: 1GB >> performance.cache-refresh-timeout: 60 >> nfs.port: 2049 >> >> >> Indeed values didn't converged automaticaly for big folder, for small >> folder like /test and /psa it was fine (I waited 1 day before checking the >> value). I was forced to run ls -lRa several times, but even with this >> workaround, values for big folder like /soft are not rightfully calculated : >> >> >> gluster volume quota venus list >> path limit_set size >> >> ---------------------------------------------------------------------------------- >> /test 100MB 4.5MB <= OK >> /psa 200MB 167.0MB <=OK >> / 7GB 5.2GB <=Strange >> /soft 5GB 4.7GB <= Strange >> >> => the size of the entire filesystem is 1.4GB >> >> >> >> Anthony >> >> >> ------------------------------ >> From: junaid at gluster.com >> Date: Mon, 5 Sep 2011 14:38:06 +0530 >> >> Subject: Re: quota calculation gluster 3.2.3 >> To: sokar6012 at hotmail.com >> CC: gluster-users at gluster.org >> >> Hi Anthony, >> >> By volume configuration, I meant 'is the volume setup dht, afr, or >> dht-afr', I would like to know the output of >> >> volume info volume-name >> >> When data is already present before the activation of quota, it takes some >> time for quota values to converge because it has to crawl through the entire >> file-system to calculate the sizes. This is automatically done when you run >> >> volume quota vol-name enable >> >> The output of >> volume quota vol-name list >> >> will reflect the aggregated value at that point in time. If the values are >> not converging then its a issue. A simple work-around is to run ls -lR on >> the mount point. Please confirm if the values were not converging >> automatically, we will have a look into it. >> >> Junaid >> >> >> On Mon, Sep 5, 2011 at 12:51 PM, anthony garnier <sokar6012 at hotmail.com>wrote: >> >> Junaid, >> >> What do you mean by " the volume configuration on other two servers" ? >> Configuration is the same on all the server.. >> I'm creating data with "dd" cmd, and yes, data was already present before >> activation of quota. >> >> >> >> ------------------------------ >> From: junaid at gluster.com >> Date: Fri, 2 Sep 2011 22:55:02 +0530 >> >> Subject: Re: quota calculation gluster 3.2.3 >> To: sokar6012 at hotmail.com >> CC: gluster-users at gluster.org >> >> Hi Anthony, >> >> Thanks for providing the input. What is the volume configuration on other >> two servers (yval1000 and yval1010)? My question is same, what data where >> you creating on these two machines? Was the data already present before >> quota was enabled? >> >> On Fri, Sep 2, 2011 at 1:49 PM, anthony garnier <sokar6012 at hotmail.com>wrote: >> >> >> I've done a ls -lRa throught the mount point and now i get : >> /soft/venus # gluster volume quota venus list >> path limit_set size >> >> ---------------------------------------------------------------------------------- >> / 2GB 1.3GB >> /test 100MB 8.0KB >> /psa 200MB 151.0MB >> /soft 1500MB 826.3MB >> >> =>Seems to be consistent >> >> and now command getfattr on yval1000 and yval1010 show : >> >> yval1000 : >> getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/ >> trusted.glusterfs.quota.size=0x0000000028f9ca00 >> >> >> getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/soft >> trusted.glusterfs.quota.size=0x000000001421d400 >> >> >> getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/psa >> trusted.glusterfs.quota.size=0x00000000053d6000 >> >> >> yval1010 : >> getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/ >> trusted.glusterfs.quota.size=0x000000002904ea00 >> >> >> getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/soft >> trusted.glusterfs.quota.size=0x000000002292ba00 >> >> >> getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/psa >> trusted.glusterfs.quota.size=0x00000000053d6000 >> >> Regards, >> >> Anthony >> >> >> ------------------------------ >> From: sokar6012 at hotmail.com >> To: junaid at gluster.com >> CC: gluster-users at gluster.org >> Subject: RE: quota calculation gluster 3.2.3 >> Date: Fri, 2 Sep 2011 07:57:19 +0000 >> >> >> I forgot to check on the 2 other server. >> So ylal3020 and ylal3030 show similar output send previously. >> >> But yval1000 and yval1010 show with all command : >> trusted.glusterfs.quota.size=0x0000000000000000 >> >> Is it normal ? >> >> Regards, >> >> Anthony >> ------------------------------ >> From: sokar6012 at hotmail.com >> To: junaid at gluster.com >> CC: gluster-users at gluster.org >> Subject: RE: quota calculation gluster 3.2.3 >> Date: Fri, 2 Sep 2011 07:51:52 +0000 >> >> Hi Junaid, >> >> Here is the output of the diffrent command executed on both server. >> Command show identical output on both server >> >> # getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus >> getfattr: Removing leading '/' from absolute path names >> # file: soft/venus >> trusted.glusterfs.quota.size=0x0000000022ccac00 >> >> # getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/test >> getfattr: Removing leading '/' from absolute path names >> # file: soft/venus/test >> trusted.glusterfs.quota.size=0x0000000000000000 >> >> # getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/psa >> getfattr: Removing leading '/' from absolute path names >> # file: soft/venus/psa >> trusted.glusterfs.quota.size=0x000000000013d000 >> >> # getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/soft >> getfattr: Removing leading '/' from absolute path names >> # file: soft/venus/soft >> trusted.glusterfs.quota.size=0x000000000d15ce00 >> >> >> # gluster volume quota venus list >> path limit_set size >> >> ---------------------------------------------------------------------------------- >> / 2GB 556.8MB >> /test 100MB 0Bytes >> /psa 200MB 1.2MB >> /soft 1500MB 209.4MB >> >> >> Indeed there is 90% of small file (~60000 files). >> >> Thx >> >> Anthony >> >> >> ------------------------------ >> From: junaid at gluster.com >> Date: Fri, 2 Sep 2011 06:18:30 +0530 >> Subject: Re: quota calculation gluster 3.2.3 >> To: sokar6012 at hotmail.com >> CC: gluster-users at gluster.org >> >> Hi Anthony, >> >> To debug this further, can you send the output of >> >> getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus >> getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/test >> getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/psa >> getfattr -n trusted.glusterfs.quota.size -e hex /soft/venus/soft >> >> from both the machine. Also check the output of >> >> gluster volume quota Replicate list >> >> Sometimes it takes a small amount of time to bring up the sizes. Also, >> what kind of data where you creating (I mean large amount of small files or >> large files because in case of small files, the directory sizes are not >> accounted by quota to calculate the size unlike the du -h command which uses >> the directory size as well). >> >> Junaid >> On Thu, Sep 1, 2011 at 1:37 PM, anthony garnier <sokar6012 at hotmail.com>wrote: >> >> Hi all, >> >> I've enable quota but I'm a bit confused by values displayed by GlusterFS >> >> Here is my volume : >> >> Volume Name: venus >> Type: Replicate >> Status: Started >> Number of Bricks: 2 >> Transport-type: tcp >> Bricks: >> Brick1: ylal3020:/soft/venus >> Brick2: ylal3030:/soft/venus >> Options Reconfigured: >> features.limit-usage: /test:100MB,/psa:200MB,/soft:1500MB,/:2GB >> features.quota: on >> diagnostics.client-log-level: ERROR >> diagnostics.brick-log-level: ERROR >> network.ping-timeout: 10 >> performance.cache-size: 2GB >> nfs.port: 2049 >> >> I've got 3 folders in the backend (/soft/venus) : >> psa 160MB (with du -sh) >> soft 1.2GB >> test 12KB >> Total 1.4GB >> >> But when I list the quota with gluster I got : >> # gluster volume quota venus list >> path limit_set size >> >> ---------------------------------------------------------------------------------- >> /test 100MB 12.0KB <= >> This one is OK >> /psa 200MB 64.4MB <= not OK >> /soft 1500MB 281.8MB <= not OK >> / 2GB 346.2MB <= not >> OK >> >> Any idea ? >> >> Regards, >> >> Anthony Garnier >> >> >> >> >> _______________________________________________ >> Gluster-users mailing list >> Gluster-users at gluster.org >> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users >> >> >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://gluster.org/pipermail/gluster-users/attachments/20110905/3513075c/attachment-0001.htm>
