Re: Problem with GlusterFS quotas (Debian Wheezy/x86_64)

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hello Raghavendra,

Thank you for your replay /and appropriate patch-set(s) :)/. Let me summarize the results I achieved during the testing. Testing environment was the same: two nodes, one replicated volume. Here is brief description of my test-bet volume.

=== snip ===
muzzy:~# gluster volume info m2d

Volume Name: m2d
Type: Replicate
Volume ID: 4d29926d-38d7-4b41-b500-83875ff37c44
Status: Started
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: g0:/g0
Brick2: g1:/g1
Options Reconfigured:
auth.allow: 194.12.*,130.204.198.50,82.119.73.118
features.quota: on
features.limit-usage: /dav/64mb:64MB,/cifs/64mb:64MB,/cifs/384mb:384MB,/cifs/352mb:352MB

muzzy:~# gluster volume quota m2d list
    path          limit_set         size
----------------------------------------------------------------------------------
/dav/64mb                  64MB               30.2MB
/cifs/64mb                 64MB               42.2MB
/cifs/384mb               384MB              365.5MB
/cifs/352mb               352MB              335.3MB

g0:
muzzy:~# du -sh /g0/dav/64mb
31M    /g0/dav/64mb
muzzy:~# du -sh /g0/cifs/64mb
43M    /g0/cifs/64mb
muzzy:~# du -sh /g0/cifs/384mb
366M    /g0/cifs/384mb
muzzy:~# du -sh /g0/cifs/352mb
336M    /g0/cifs/352mb

g1:
root@g1:/# du -sh /g1/dav/64mb
31M    /g1/dav/64mb
root@g1:/# du -sh /g1/cifs/64mb
43M    /g1/cifs/64mb
root@g1:/# du -sh /g1/cifs/384mb
366M    /g1/cifs/384mb
root@g1:/# du -sh /g1/cifs/352mb
336M    /g1/cifs/352mb
=== snip ===

Basically I did again my quota testing in both (already described) ways, i.e. with quota limit set on existing and on non-existing directory entry. Everything went ok this time. Quota limits correctly take effect and client sides are notified correctly.

Let me describe some more findings during the testing:

(o) patches applied to master glusterfs branch:

- a compile time issue: after execution of autogen.sh script, where was successfully generated configure scrip. When I tried to use the script, the following error occured:

=== snip ===
checking if libxml2 is present... ./configure: line 12771: syntax error near unexpected token `LIBXML2,'
./configure: line 12771: `PKG_CHECK_MODULES(LIBXML2, libxml-2.0 >= 2.6.19,'
=== snip ===

After fixing broken check, I had successful compilation and installation. During, my testing with master branch, I noticed the following:

- there was a huge slow down of file transfer between g0 serving node and my testing webdav client;
- there was a strange quota size variation during the upload;
- although these problems quota limit restriction successfully took place;

(o) patches applied to qa37 tar:

- no compile time issues;
- i did not notice any performance issues;
- quota lime worked like expected;

So, for now I'm stuck on 3.3.0qa37 + described patches :)

Please find attached glusterd logs from two nodes (g0 and g1) and native client log from g0 node. All logs are taken with DEBUG option on.

I've planed more tests - with distributed and replicated volume. I'll test again with patched 3.3.0qa37 version and I hope to achieve same results :) during the testing.

Regards,
Hristo Hristov

On 4/25/12 5:14 AM, Raghavendra Gowdappa wrote:
Hi,

Can you apply following patch to latest master?
http://review.gluster.com/3022

and let us know, whether it solves your issue?

If you want to stick to 3.3.0qa37, you need following patch along with the above:
http://review.gluster.com/#change,3021

regards,
Raghavendra.

----- Original Message -----
From: "Hristo Hristov"<id@xxxxxxxx>
To: "Vijay Bellur"<vbellur@xxxxxxxxxx>
Cc: gluster-devel@xxxxxxxxxx, "Raghavendra Gowdappa"<rgowdapp@xxxxxxxxxx>
Sent: Monday, April 23, 2012 7:25:35 PM
Subject: Re: Problem with GlusterFS quotas (Debian Wheezy/x86_64)

Hello Vijay,

I made the testing against the version 3.3.0 qa 37 of gluster fs with
same cluster config. I'm pasting configuration info:

=== snip ===
muzzy:/home/lisp/work/fs/logs# gluster --version
glusterfs 3.3.0qa37 built on Apr 23 2012 13:55:08
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2011 Gluster Inc.<http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU
General Public License.

muzzy:/home/lisp/work/fs/logs# gluster volume info m2d

Volume Name: m2d
Type: Replicate
Volume ID: 752e6b5b-504b-46bc-be5e-498fc5f364d1
Status: Started
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: g0:/g0
Brick2: g1:/g1
Options Reconfigured:
features.limit-usage:
/dav/64mb:64MB,/cifs/64mb:64MB,/dav/92mb:92MB,/cifs/92mb:92MB,/cifs/128mb:128MB,/dav/128mb:128MB
features.quota: on
auth.allow: 194.12.*
muzzy:/home/lisp/work/fs/logs#

muzzy:/home/lisp/work/fs/logs# gluster volume quota m2d list
       path          limit_set         size
----------------------------------------------------------------------------------
/dav/64mb                  64MB              365.5MB
/cifs/64mb                 64MB              365.5MB
/dav/92mb                  92MB              365.5MB
/cifs/92mb                 92MB               0Bytes
/cifs/128mb               128MB              126.9MB
/dav/128mb                128MB               83.6MB


mount -t glusterfs g0:/m2d /mnt/m2d -o
acl,log-level=DEBUG,log-file=/usr/local/var/log/glusterfs/g0-mount-client.log
=== snip ===

Unfortunately, gluster's quota management behaves exactly the same
way,
like in previous version. When I set a quota limit (no matter on
existing or not existing directory), quota limitation doesn't work,
until net next gluster volume remount (i.e. unmount and then mount)

Please find attached log files from g0 node:

(o) g0-daemon.log
log from glusterfs daemon, run in debug mode;

(o) 0-mount-client.log.00.before-remounting
client log before remounting;

(o) g0-mount-client.log.01.after-remounting
client log after unmount, mount (i.e. remounting) sequence; then
quota
restrictions work as expected (Entries '/cifs/128mb' and '/dav/128mb'
in
upper 'gluster volume quota m2d list' quota).

All the log files are inside 3-3-0-qa-38-logs.tar.gz archive.

One more thing, could you please point me out something like a
'big-picture-of-glusterfs-architecture' page/doc @ gluster's web
site.
I'm constantly finding a
http://www.gluster.org/community/documentation/index.php/GlusterFS_Hackers_Guide
link, which is quite outdated :(

Regards,
Hristo Hristov

On 4/23/12 1:26 PM, Vijay Bellur wrote:
On 04/23/2012 01:17 PM, Hristo Hristov wrote:
Thank you for your replay. I'm sending you requested files, please
correct me if there is a missing one :) /debian logs its services
in
debianish way, so I may be missing something:)/

(o) glusterd log from brick 0:

location: /var/log/glusterfs/bricks
name: g0.log

(o) native client mount log

name: gluster.log

I'll definitely try v3.3.0 release of glusterfs with same test
configuration. Which version I should try (i.e. which one is more
recent):

(o)
http://bits.gluster.com/pub/gluster/glusterfs/src/glusterfs-3.3.0qa37.tar.gz

qa37 is more recent. That would be a good release to try this out
on.

Thanks,
Vijay




Attachment: qa37.patched.tar.gz
Description: GNU Zip compressed data


[Index of Archives]     [Gluster Users]     [Ceph Users]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Security]     [Bugtraq]     [Linux]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux