Re: Fwd: to mailing list gluster.org

Pranith Kumar Karampuri <pkarampu@xxxxxxxxxx> · Thu, 22 Jan 2015 00:46:34 +0530



    + quota devs

    
    Pranith

    On 01/07/2015 07:37 PM, Bekk, Klaus
      (IKP) wrote:

    
            Hi all,
             
            we are running 4
                server with Ubuntu 14.04 as glusterfs server with
                3.5.2-ubuntu1~trusty1.
          
        
          When starting quotas on any
                of the volumes the brick will be not available after
                some time. So we have to disable quota on this volume
                and stop and start this volume.
           
        
            We have the
                following configuration
             
            root@ikpsrv01:/var/log/glusterfs#
                gluster volume info
             
            Volume Name: gltestn
            Type: Distribute
            Volume ID:
                e50bd30d-dbc8-4279-ac9d-2e557328a643
            Status: Started
            Number of Bricks: 1
            Transport-type: tcp
            Bricks:
            Brick1:
                ikpsrv01:/gltest2/gl
             
            Volume Name: glwww1
            Type: Replicate
            Volume ID:
                811e4c2a-4520-4e61-bb84-955afbdb9dff
            Status: Started
            Number of Bricks: 1
                x 2 = 2
            Transport-type: tcp
            Bricks:
            Brick1:
                ikpsrv03:/glwwwn/gl
            Brick2:
                ikpsrv01:/glwwwn2/gl
            Options
                Reconfigured:
            features.quota: on
             
            Volume Name: corsika
            Type: Distribute
            Volume ID:
                d6e07f3e-4b35-4736-9b71-afc3224f0f29
            Status: Started
            Number of Bricks: 4
            Transport-type: tcp
            Bricks:
            Brick1:
                ikpsrv03:/glcors3-1/gl
            Brick2:
                ikpsrv03:/glcors3-2/gl
            Brick3:
                ikpsrv02:/glcors2-1/gl
            Brick4:
                ikpsrv02:/glcors2-2/gl
             
            Volume Name: gldata
            Type: Distribute
            Volume ID:
                82485d03-9ef4-4151-a2cc-881072e871a1
            Status: Started
            Number of Bricks: 2
            Transport-type: tcp
            Bricks:
            Brick1:
                ikpsrv02:/gldata2-1/gl
            Brick2:
                ikpsrv01:/gldata1-2/gl
            Options
                Reconfigured:
            features.quota: on
             
            Volume Name: gltest
            Type: Distribute
            Volume ID:
                9e69afcc-d421-41c9-824b-7d9a31240072
            Status: Started
            Number of Bricks: 2
            Transport-type: tcp
            Bricks:
            Brick1:
                ikpsrv01:/gltest3/gl
            Brick2:
                ikpsrv01:/gltest4/gl
            Options
                Reconfigured:
            features.quota: off
             
            Volume Name:
                glusersold
            Type: Distribute
            Volume ID:
                91a79e1a-4ff3-4191-b0ae-73357105774c
            Status: Started
            Number of Bricks: 1
            Transport-type: tcp
            Bricks:
            Brick1:
                ikpsrv02:/glusers2-2/gl
             
            Volume Name: glusers
            Type: Distribute
            Volume ID:
                f48b1941-9c19-4311-bd9e-497660fbcc80
            Status: Started
            Number of Bricks: 1
            Transport-type: tcp
            Bricks:
            Brick1:
                ikpsrv02:/glusers2-1/gl
            Options
                Reconfigured:
            diagnostics.client-log-level:
                CRITICAL
            diagnostics.brick-log-level:
                CRITICAL
            features.quota: off
             
            When starting quotas
                on any of the volumes the brick will be not available
                after some time. So we have to disable quota on this
                volume and stop and start this volume.
             
            The command 
             
            Gluster volume quota
                gltestn enable 
            gives the following
                output in quotad.log
             
             
            [2015-01-07 12:01:55.522229] W
              [glusterfsd.c:1095:cleanup_and_exit]
              (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)
              [0x7fb455bd9fbd]
              (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182)
              [0x7fb455ead182]
              (-->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xd5)
              [0x7fb4569a7265]))) 0-: received signum (15), shutting
              down
            [2015-01-07
                12:01:56.534931] I [glusterfsd.c:1959:main]
                0-/usr/sbin/glusterfs: Started running
                /usr/sbin/glusterfs version 3.5.2 (/usr/sbin/glusterfs
                -s localhost --volfile-id gluster/quotad -p
                /var/lib/glusterd/quotad/run/quotad.pid -l
                /var/log/glusterfs/quotad.log -S
                /var/run/fa869d5552bf340ec7506747e21b2841.socket
                --xlator-option *replicate*.data-self-heal=off
                --xlator-option *replicate*.metadata-self-heal=off
                --xlator-option *replicate*.entry-self-heal=off)
            [2015-01-07
                12:01:56.537061] I [socket.c:3561:socket_init]
                0-socket.glusterfsd: SSL support is NOT enabled
            [2015-01-07
                12:01:56.537134] I [socket.c:3576:socket_init]
                0-socket.glusterfsd: using system polling thread
            [2015-01-07
                12:01:56.537355] I [socket.c:3561:socket_init]
                0-glusterfs: SSL support is NOT enabled
            [2015-01-07
                12:01:56.537383] I [socket.c:3576:socket_init]
                0-glusterfs: using system polling thread
            [2015-01-07
                12:01:56.541480] I [graph.c:254:gf_add_cmdline_options]
                0-glwww1-replicate-0: adding option 'entry-self-heal'
                for volume 'glwww1-replicate-0' with value 'off'
            [2015-01-07
                12:01:56.541516] I [graph.c:254:gf_add_cmdline_options]
                0-glwww1-replicate-0: adding option 'metadata-self-heal'
                for volume 'glwww1-replicate-0' with value 'off'
            [2015-01-07
                12:01:56.541536] I [graph.c:254:gf_add_cmdline_options]
                0-glwww1-replicate-0: adding option 'data-self-heal' for
                volume 'glwww1-replicate-0' with value 'off'
            [2015-01-07
                12:01:56.543122] I [socket.c:3561:socket_init]
                0-socket.quotad: SSL support is NOT enabled
            [2015-01-07
                12:01:56.543157] I [socket.c:3576:socket_init]
                0-socket.quotad: using system polling thread
            [2015-01-07
                12:01:56.543404] I [dht-shared.c:311:dht_init_regex]
                0-glwww1: using regex rsync-hash-regex = ^\.(.+)\.[^.]+$
            [2015-01-07
                12:01:56.547124] I [socket.c:3561:socket_init]
                0-glwww1-client-1: SSL support is NOT enabled
            [2015-01-07
                12:01:56.547158] I [socket.c:3576:socket_init]
                0-glwww1-client-1: using system polling thread
            [2015-01-07
                12:01:56.547728] I [socket.c:3561:socket_init]
                0-glwww1-client-0: SSL support is NOT enabled
            [2015-01-07
                12:01:56.547754] I [socket.c:3576:socket_init]
                0-glwww1-client-0: using system polling thread
            [2015-01-07
                12:01:56.547820] I [dht-shared.c:311:dht_init_regex]
                0-gltestn: using regex rsync-hash-regex =
                ^\.(.+)\.[^.]+$
            [2015-01-07
                12:01:56.549471] I [socket.c:3561:socket_init]
                0-gltestn-client-0: SSL support is NOT enabled
            [2015-01-07
                12:01:56.549501] I [socket.c:3576:socket_init]
                0-gltestn-client-0: using system polling thread
            [2015-01-07
                12:01:56.549565] I [dht-shared.c:311:dht_init_regex]
                0-gldata: using regex rsync-hash-regex = ^\.(.+)\.[^.]+$
            [2015-01-07
                12:01:56.551155] I [socket.c:3561:socket_init]
                0-gldata-client-1: SSL support is NOT enabled
            [2015-01-07
                12:01:56.551184] I [socket.c:3576:socket_init]
                0-gldata-client-1: using system polling thread
            [2015-01-07
                12:01:56.551744] I [socket.c:3561:socket_init]
                0-gldata-client-0: SSL support is NOT enabled
            [2015-01-07
                12:01:56.551770] I [socket.c:3576:socket_init]
                0-gldata-client-0: using system polling thread
            [2015-01-07
                12:01:56.551790] W [graph.c:329:_log_if_unknown_option]
                0-quotad: option 'rpc-auth.auth-glusterfs' is not
                recognized
            [2015-01-07
                12:01:56.551819] W [graph.c:329:_log_if_unknown_option]
                0-quotad: option 'rpc-auth.auth-unix' is not recognized
            [2015-01-07
                12:01:56.551838] W [graph.c:329:_log_if_unknown_option]
                0-quotad: option 'rpc-auth.auth-null' is not recognized
            [2015-01-07
                12:01:56.551854] W [graph.c:329:_log_if_unknown_option]
                0-quotad: option 'glwww1.volume-id' is not recognized
            [2015-01-07
                12:01:56.551870] W [graph.c:329:_log_if_unknown_option]
                0-quotad: option 'gltestn.volume-id' is not recognized
            [2015-01-07
                12:01:56.551885] W [graph.c:329:_log_if_unknown_option]
                0-quotad: option 'gldata.volume-id' is not recognized
            [2015-01-07
                12:01:56.551922] I [client.c:2294:notify]
                0-glwww1-client-0: parent translators are ready,
                attempting connect on transport
            [2015-01-07
                12:01:56.552579] I [client.c:2294:notify]
                0-glwww1-client-1: parent translators are ready,
                attempting connect on transport
            [2015-01-07
                12:01:56.553179] I [client.c:2294:notify]
                0-gltestn-client-0: parent translators are ready,
                attempting connect on transport
            [2015-01-07
                12:01:56.553757] I [client.c:2294:notify]
                0-gldata-client-0: parent translators are ready,
                attempting connect on transport
            [2015-01-07
                12:01:56.554372] I [client.c:2294:notify]
                0-gldata-client-1: parent translators are ready,
                attempting connect on transport
            Final graph:
            +------------------------------------------------------------------------------+
              1: volume
                glwww1-client-0
              2:     type
                protocol/client
              3:     option
                remote-host ikpsrv03
              4:     option
                remote-subvolume /glwwwn/gl
              5:     option
                transport-type socket
              6:    
              option username 7c5931b4-d41e-40f7-978e-4b5a40f49481
              7:     option
                password 6ab1661b-4752-42a4-80cc-ca0bd0167a9f
              8: end-volume
              9:
             10: volume
                glwww1-client-1
             11:     type
                protocol/client
             12:     option
                remote-host ikpsrv01
             13:     option
                remote-subvolume /glwwwn2/gl
             14:     option
                transport-type socket
             15:    
              option username 7c5931b4-d41e-40f7-978e-4b5a40f49481
             16:     option
                password 6ab1661b-4752-42a4-80cc-ca0bd0167a9f
             17: end-volume
             18:
             19: volume
                glwww1-replicate-0
             20:     type
                cluster/replicate
             21:     option
                data-self-heal off
             22:     option
                metadata-self-heal off
             23:     option
                entry-self-heal off
             24:     subvolumes
                glwww1-client-0 glwww1-client-1
             25: end-volume
             26:
             27: volume glwww1
             28:     type
                cluster/distribute
             29:     subvolumes
                glwww1-replicate-0
             30: end-volume
             31:
             32: volume
                gltestn-client-0
             33:     type
                protocol/client
             34:     option
                remote-host ikpsrv01
             35:     option
                remote-subvolume /gltest2/gl
             36:     option
                transport-type socket
             37:     option
                username 76c8bf85-a5c5-445d-8e04-e596b6eb173b
             38:     option
                password ceec81fd-5afd-4d6b-a413-00b2688f4c62
             39: end-volume
             40:
             41: volume gltestn
             42:     type
                cluster/distribute
             43:     subvolumes
                gltestn-client-0
             44: end-volume
             45:
             46: volume
                gldata-client-0
             47:     type
                protocol/client
             48:     option
                remote-host ikpsrv02
             49:     option
                remote-subvolume /gldata2-1/gl
             50:     option
                transport-type socket
             51:    
              option username e9dcdbec-1110-4ab2-b667-63625126b246
             52:     option
                password 05a9e37d-67a0-476b-88df-93d612a4e08e
             53: end-volume
             54:
             55: volume
                gldata-client-1
             56:     type
                protocol/client
             57:     option
                remote-host ikpsrv01
             58:     option
                remote-subvolume /gldata1-2/gl
             59:     option
                transport-type socket
             60:    
              option username e9dcdbec-1110-4ab2-b667-63625126b246
             61:     option
                password 05a9e37d-67a0-476b-88df-93d612a4e08e
             62: end-volume
             63:
             64: volume gldata
             65:     type
                cluster/distribute
             66:     subvolumes
                gldata-client-0 gldata-client-1
             67: end-volume
             68:
             69: volume quotad
             70:     type
                features/quotad
             71:     option
                rpc-auth.auth-glusterfs on
             72:     option
                rpc-auth.auth-unix on
             73:     option
                rpc-auth.auth-null on
             74:     option
                transport.socket.listen-path /tmp/quotad.socket
             75:     option
                transport-type socket
             76:     option
                transport.address-family unix
             77:     option
                glwww1.volume-id glwww1
             78:     option
                gltestn.volume-id gltestn
             79:     option
                gldata.volume-id gldata
             80:     subvolumes
                glwww1 gltestn gldata
             81: end-volume
             82:
            +------------------------------------------------------------------------------+
            [2015-01-07
                12:01:56.555618] I [rpc-clnt.c:1729:rpc_clnt_reconfig]
                0-gldata-client-0: changing port to 49160 (from 0)
            [2015-01-07
                12:01:56.556529] I [rpc-clnt.c:1729:rpc_clnt_reconfig]
                0-glwww1-client-1: changing port to 49165 (from 0)
            [2015-01-07
                12:01:56.556597] I [rpc-clnt.c:1729:rpc_clnt_reconfig]
                0-gltestn-client-0: changing port to 49166 (from 0)
            [2015-01-07
                12:01:56.556643] I [rpc-clnt.c:1729:rpc_clnt_reconfig]
                0-gldata-client-1: changing port to 49178 (from 0)
            [2015-01-07
                12:01:56.558382] W
                [client-callback.c:31:client_cbk_fetchspec]
                0-glwww1-client-0: this function should not be called
            [2015-01-07
                12:01:56.558484] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec]
                0-mgmt: Volume file changed
            [2015-01-07
                12:01:56.558601] I
                [client-handshake.c:1677:select_server_supported_programs]
                0-gldata-client-0: Using Program GlusterFS 3.3, Num
                (1298437), Version (330)
            [2015-01-07
                12:01:56.559594] I
                [client-handshake.c:1677:select_server_supported_programs]
                0-gltestn-client-0: Using Program GlusterFS 3.3, Num
                (1298437), Version (330)
            [2015-01-07
                12:01:56.559686] I
                [client-handshake.c:1677:select_server_supported_programs]
                0-glwww1-client-1: Using Program GlusterFS 3.3, Num
                (1298437), Version (330)
            [2015-01-07
                12:01:56.559768] I
                [client-handshake.c:1677:select_server_supported_programs]
                0-gldata-client-1: Using Program GlusterFS 3.3, Num
                (1298437), Version (330)
            [2015-01-07
                12:01:56.559852] I
                [client-handshake.c:1462:client_setvolume_cbk]
                0-gldata-client-0: Connected to 141.52.67.86:49160,
                attached to remote volume '/gldata2-1/gl'.
            [2015-01-07
                12:01:56.559873] I
                [client-handshake.c:1474:client_setvolume_cbk]
                0-gldata-client-0: Server and Client lk-version numbers
                are not same, reopening the fds
            [2015-01-07
                12:01:56.559996] I
                [client-handshake.c:1462:client_setvolume_cbk]
                0-gltestn-client-0: Connected to 141.52.67.85:49166,
                attached to remote volume '/gltest2/gl'.
            [2015-01-07
                12:01:56.560017] I
                [client-handshake.c:1474:client_setvolume_cbk]
                0-gltestn-client-0: Server and Client lk-version numbers
                are not same, reopening the fds
            [2015-01-07
                12:01:56.560089] I
                [client-handshake.c:450:client_set_lk_version_cbk]
                0-gldata-client-0: Server lk version = 1
            [2015-01-07
                12:01:56.560139] I
                [client-handshake.c:1462:client_setvolume_cbk]
                0-glwww1-client-1: Connected to 141.52.67.85:49165,
                attached to remote volume '/glwwwn2/gl'.
            [2015-01-07
                12:01:56.560158] I
                [client-handshake.c:1474:client_setvolume_cbk]
                0-glwww1-client-1: Server and Client lk-version numbers
                are not same, reopening the fds
            [2015-01-07
                12:01:56.560204] I [afr-common.c:4131:afr_notify]
                0-glwww1-replicate-0: Subvolume 'glwww1-client-1' came
                back up; going online.
            [2015-01-07
                12:01:56.560262] I
                [client-handshake.c:1462:client_setvolume_cbk]
                0-gldata-client-1: Connected to 141.52.67.85:49178,
                attached to remote volume '/gldata1-2/gl'.
            [2015-01-07
                12:01:56.560301] I
                [client-handshake.c:1474:client_setvolume_cbk]
                0-gldata-client-1: Server and Client lk-version numbers
                are not same, reopening the fds
            [2015-01-07
                12:01:56.560385] I
                [client-handshake.c:450:client_set_lk_version_cbk]
                0-gltestn-client-0: Server lk version = 1
            [2015-01-07
                12:01:56.560467] I
                [client-handshake.c:450:client_set_lk_version_cbk]
                0-glwww1-client-1: Server lk version = 1
            [2015-01-07
                12:01:56.560546] I
                [client-handshake.c:450:client_set_lk_version_cbk]
                0-gldata-client-1: Server lk version = 1
            [2015-01-07
                12:01:57.255911] I
                [glusterfsd-mgmt.c:1307:mgmt_getspec_cbk] 0-glusterfs:
                No change in volfile, continuing
            [2015-01-07
                12:01:57.557002] I [rpc-clnt.c:1729:rpc_clnt_reconfig]
                0-glwww1-client-0: changing port to 49153 (from 0)
            [2015-01-07
                12:01:57.557924] I
                [client-handshake.c:1677:select_server_supported_programs]
                0-glwww1-client-0: Using Program GlusterFS 3.3, Num
                (1298437), Version (330)
            [2015-01-07
                12:01:57.558360] I
                [client-handshake.c:1462:client_setvolume_cbk]
                0-glwww1-client-0: Connected to 141.52.67.96:49153,
                attached to remote volume '/glwwwn/gl'.
            [2015-01-07
                12:01:57.558405] I
                [client-handshake.c:1474:client_setvolume_cbk]
                0-glwww1-client-0: Server and Client lk-version numbers
                are not same, reopening the fds
            [2015-01-07
                12:01:57.558706] I
                [client-handshake.c:450:client_set_lk_version_cbk]
                0-glwww1-client-0: Server lk version = 1
             
            what means 
             
            2015-01-07
                12:01:56.559594] I
                [client-handshake.c:1677:select_server_supported_programs]
                0-gltestn-client-0: Using Program GlusterFS 3.3, Num
                (1298437), Version (330)
             
            In the log output.
             
            Our glusterfs
                clients all have 3.5.2 glusterfs version.
             
            Can someone help?
             
            kind regards
            Klaus
                Bekk
             
             
            ----------------------------------------------------------
            Karlsruher
                Institut of Technologie (KIT)
            Institut
                für Kernphysik (IKP)
             
            Dr.
                Klaus Bekk
             
            Hermann-von-Helmholtz-Platz
                1
            Campus
                Nord Gebäude 425
            76344
                Eggenstein-Leopoldshafen
             
            Telefon:
                +49 721 608-23382
            Fax:    
                +49 721 608-23321
             
            E-Mail:
                Klaus.Bekk@xxxxxxx
            http://www.kit.edu/
             
            KIT – University of the State of
                Baden-Wuerttemberg and
            National Research Center of the Helmholtz
                Association
            ----------------------------------------------------------
             
             
      _______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users
    
    
_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users