Dear list, I forgot to add info about extended fs attributes for directories with quota in my glusterfs setup. Here it is: === snip === (o) muzzy, aka g0 muzzy:~# getfattr -m . -d -e hex /g0/dav/128mb getfattr: Removing leading '/' from absolute path names # file: g0/dav/128mb trusted.gfid=0xd226869ff0ce4e519403be39a5842ebf trusted.glusterfs.dht=0x0000000100000000aaaaaaaaffffffff trusted.glusterfs.quota.c4218eca-9bbf-468d-8dba-3f2f135d8f38.contri=0x0000000000003000 trusted.glusterfs.quota.dirty=0x3000 trusted.glusterfs.quota.size=0x0000000000003000 root at g1:~# getfattr -m . -d -e hex /g1/dav/128mb getfattr: Removing leading '/' from absolute path names # file: g1/dav/128mb trusted.gfid=0xd226869ff0ce4e519403be39a5842ebf trusted.glusterfs.dht=0x00000001000000000000000055555554 trusted.glusterfs.quota.c4218eca-9bbf-468d-8dba-3f2f135d8f38.contri=0x000000000539f000 trusted.glusterfs.quota.dirty=0x3000 trusted.glusterfs.quota.size=0x000000000539f000 root at mdx:~# getfattr -m . -d -e hex /g2/dav/128mb getfattr: Removing leading '/' from absolute path names # file: g2/dav/128mb trusted.gfid=0xd226869ff0ce4e519403be39a5842ebf trusted.glusterfs.dht=0x000000010000000055555555aaaaaaa9 trusted.glusterfs.quota.c4218eca-9bbf-468d-8dba-3f2f135d8f38.contri=0x0000000000004000 trusted.glusterfs.quota.dirty=0x3000 trusted.glusterfs.quota.size=0x0000000000004000 === snip === Also, I'm listing all the quotas added in my setup: === snip === muzzy:~# gluster volume quota m3d list path limit_set size ---------------------------------------------------------------------------------- /1Gg 1GB /10Gg 10GB /5Gg 5GB /512Mb 512MB 0Bytes /dav/256MB 256MB 208.8MB /dav/1GB 1GB 0Bytes /dav/5GB 5GB 0Bytes /dav/10GB 10GB 0Bytes /cifs/256Mb 256MB 167.5MB /dav/128mb 128MB 83.6MB /dav/192mb 192MB 365.4MB /cifs/192mb 192MB 210.9MB /cifs/64mb 64MB 365.4MB === snip === Any ideas are welcome, and 10x for your attention. Regards, Hristo Hristov On 4/19/12 5:21 PM, Hristo Hristov wrote: > Hello list, > > I'm experimenting with a little GlusterFS cluster on debian wheezy: > > === snip === > muzzy:~# cat /etc/debian_version > wheezy/sid > > muzzy:~# dpkg -l | grep gluster > ii glusterfs-client 3.2.6-1 clustered file-system (client package) > ii glusterfs-common 3.2.6-1 GlusterFS common libraries and translator > modules > ii glusterfs-server 3.2.6-1 clustered file-system (server package) > === snip === > > My volume has 3 bricks and is a distributed one: > > === snip === > muzzy:~# gluster volume info > > Volume Name: m3d > Type: Distribute > Status: Started > Number of Bricks: 3 > Transport-type: tcp > Bricks: > Brick1: g0:/g0 > Brick2: g1:/g1 > Brick3: g2:/g2 > Options Reconfigured: > features.quota-timeout: 0 > features.limit-usage: > /1Gg:1GB,/10Gg:10GB,/5Gg:5GB,/512Mb:512MB,/dav/256MB:256MB,/dav/1GB:1GB,/dav/5GB:5GB,/dav/10GB:10GB,/cifs/256Mb:256MB,/dav/128mb:128MB,/dav/192mb:192MB,/cifs/192mb:192MB,/cifs/64mb:64MB > features.quota: on > auth.allow: 194.12.*,130.204.198.50 > === snip === > > One of nodes, (g0), also mounts m3d volume (mount -t glusterfs) and > "exports" it via samba and webdav. I'm using two different directories > under volume root for each of them, i.e. dav and cifs. /This is not > the best way to do export the volume, but I do not have more nodes at > the moment :(/ > > I'm mounting the volume with the following parameters: > === snip === > mount -t glusterfs g0:/m3d /mnt/m3d -o > acl,log-level=WARNING,log-file=/var/log/gluster.log > === snip === > > According the glusterfs documentation > (Gluster_File_System-3.2.5-Administration_Guide-en-US.pdf, p. 67) we > can set directory limit even on non-existing directory: > === quote === > Note > You can set the disk limit on the directory even if it is not created. > The disk limit is enforced > immediately after creating that directory. For more information on > setting disk limit, see > Section 10.3, ?Setting or Replacing Disk Limit ?. > === quote === > > Here comes my problem. When I set directory quota on non-existing (or > even existing directory) the quota limit is not enforced. I have to > unmount the volume (i.e. stop apache and samba daemons before that) > and mount it again (respectively starting back apache and samba > daemons). Then quota limit is enforced correctly on existing > directories. If I create a new directory and set quota limit, i have > to umount and mount again in order to have working quota for that > directory. > > Do you have any ideas, what may cause this problem? Will it be > appropriate to blame Debian Wheezy :) /because I'm using It's testing > branch/? > > Here is more info about my testing glusterfs installation: > === snip === > muzzy:~# cat /etc/debian_version > wheezy/sid > > muzzy:~# uname -a > Linux muzzy 3.2.0-1-amd64 #1 SMP Fri Feb 17 05:17:36 UTC 2012 x86_64 > GNU/Linux > > muzzy:~# cat /var/log/gluster.log > 2012-04-19 16:36:17.663509] W [write-behind.c:3023:init] > 0-m3d-write-behind: disabling write-behind for first 0 bytes > Given volfile: > +------------------------------------------------------------------------------+ > > 1: volume m3d-client-0 > 2: type protocol/client > 3: option remote-host g0 > 4: option remote-subvolume /g0 > 5: option transport-type tcp > 6: end-volume > 7: > 8: volume m3d-client-1 > 9: type protocol/client > 10: option remote-host g1 > 11: option remote-subvolume /g1 > 12: option transport-type tcp > 13: end-volume > 14: > 15: volume m3d-client-2 > 16: type protocol/client > 17: option remote-host g2 > 18: option remote-subvolume /g2 > 19: option transport-type tcp > 20: end-volume > 21: > 22: volume m3d-dht > 23: type cluster/distribute > 24: subvolumes m3d-client-0 m3d-client-1 m3d-client-2 > 25: end-volume > 26: > 27: volume m3d-quota > 28: type features/quota > 29: option limit-set > /1Gg:1GB,/10Gg:10GB,/5Gg:5GB,/512Mb:512MB,/dav/256MB:256MB,/dav/1GB:1GB,/dav/5GB:5GB,/dav/10GB:10GB,/cifs/256Mb:256MB,/dav/128mb:128MB,/dav/192mb:192MB,/cifs/192mb:192MB > 30: option timeout 0 > 31: subvolumes m3d-dht > 32: end-volume > 33: > 34: volume m3d-write-behind > 35: type performance/write-behind > 36: subvolumes m3d-quota > 37: end-volume > 38: > 39: volume m3d-read-ahead > 40: type performance/read-ahead > 41: subvolumes m3d-write-behind > 42: end-volume > 43: > 44: volume m3d-io-cache > 45: type performance/io-cache > 46: subvolumes m3d-read-ahead > 47: end-volume > 48: > 49: volume m3d-quick-read > 50: type performance/quick-read > 51: subvolumes m3d-io-cache > 52: end-volume > 53: > 54: volume m3d-stat-prefetch > 55: type performance/stat-prefetch > 56: subvolumes m3d-quick-read > 57: end-volume > 58: > 59: volume m3d > 60: type debug/io-stats > 61: option latency-measurement off > 62: option count-fop-hits off > 63: subvolumes m3d-stat-prefetch > 64: end-volume > > +------------------------------------------------------------------------------+ > > [2012-04-19 16:37:12.729422] W [dict.c:1153:data_to_str] > (-->/usr/lib/libglusterfs.so.0(+0x18ddc) [0x7ff9c3025ddc] > (-->/usr/lib/libglusterfs.so.0(+0x18e6a) [0x7ff9c3025e6a] > (-->/usr/lib/glusterfs/3.2.6/xlator/performance/io-cache.so(reconfigure+0x443) > [0x7ff9bf0f7913]))) 0-dict: data is NULL > [2012-04-19 16:37:12.729481] W [dict.c:1153:data_to_str] > (-->/usr/lib/libglusterfs.so.0(+0x18ddc) [0x7ff9c3025ddc] > (-->/usr/lib/libglusterfs.so.0(+0x18e6a) [0x7ff9c3025e6a] > (-->/usr/lib/glusterfs/3.2.6/xlator/performance/io-cache.so(reconfigure+0x59d) > [0x7ff9bf0f7a6d]))) 0-dict: data is NULL > [2012-04-19 16:37:12.729531] W [dict.c:1153:data_to_str] > (-->/usr/lib/libglusterfs.so.0(+0x18ddc) [0x7ff9c3025ddc] > (-->/usr/lib/libglusterfs.so.0(+0x18e6a) [0x7ff9c3025e6a] > (-->/usr/lib/glusterfs/3.2.6/xlator/performance/io-cache.so(reconfigure+0x72f) > [0x7ff9bf0f7bff]))) 0-dict: data is NULL > [2012-04-19 16:37:12.729580] W [dict.c:1153:data_to_str] > (-->/usr/lib/libglusterfs.so.0(+0x18ddc) [0x7ff9c3025ddc] > (-->/usr/lib/libglusterfs.so.0(+0x18e6a) [0x7ff9c3025e6a] > (-->/usr/lib/glusterfs/3.2.6/xlator/performance/io-cache.so(reconfigure+0x74b) > [0x7ff9bf0f7c1b]))) 0-dict: data is NULL > === snip === > > Thank you in advance, > Regards > Hristo Hristov > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users