Re: Fwd: to mailing list gluster.org

Pranith Kumar Karampuri <pkarampu@xxxxxxxxxx> · Wed, 28 Jan 2015 12:13:41 +0530



    Vijaikumar, Sachin will work on this quota issue.

    
    Pranith

    On 01/22/2015 12:46 AM, Pranith Kumar
      Karampuri wrote:

    
      + quota devs

      
      Pranith

      On 01/07/2015 07:37 PM, Bekk, Klaus
        (IKP) wrote:

      
              Hi all,
               
              we are running 4
                  server with Ubuntu 14.04 as glusterfs server with
                  3.5.2-ubuntu1~trusty1.
            
          
            When
                  starting quotas on any of the volumes the brick will
                  be not available after some time. So we have to
                  disable quota on this volume and stop and start this
                  volume.
             
          
              We have the
                  following configuration
               
              root@ikpsrv01:/var/log/glusterfs#

                  gluster volume info
               
              Volume Name:
                  gltestn
              Type: Distribute
              Volume ID:
                  e50bd30d-dbc8-4279-ac9d-2e557328a643
              Status: Started
              Number of Bricks:
                  1
              Transport-type:
                  tcp
              Bricks:
              Brick1:
                  ikpsrv01:/gltest2/gl
               
              Volume Name:
                  glwww1
              Type: Replicate
              Volume ID:
                  811e4c2a-4520-4e61-bb84-955afbdb9dff
              Status: Started
              Number of Bricks:
                  1 x 2 = 2
              Transport-type:
                  tcp
              Bricks:
              Brick1:
                  ikpsrv03:/glwwwn/gl
              Brick2:
                  ikpsrv01:/glwwwn2/gl
              Options
                  Reconfigured:
              features.quota: on
               
              Volume Name:
                  corsika
              Type: Distribute
              Volume ID:
                  d6e07f3e-4b35-4736-9b71-afc3224f0f29
              Status: Started
              Number of Bricks:
                  4
              Transport-type:
                  tcp
              Bricks:
              Brick1:
                  ikpsrv03:/glcors3-1/gl
              Brick2:
                  ikpsrv03:/glcors3-2/gl
              Brick3:
                  ikpsrv02:/glcors2-1/gl
              Brick4:
                  ikpsrv02:/glcors2-2/gl
               
              Volume Name:
                  gldata
              Type: Distribute
              Volume ID:
                  82485d03-9ef4-4151-a2cc-881072e871a1
              Status: Started
              Number of Bricks:
                  2
              Transport-type:
                  tcp
              Bricks:
              Brick1:
                  ikpsrv02:/gldata2-1/gl
              Brick2:
                  ikpsrv01:/gldata1-2/gl
              Options
                  Reconfigured:
              features.quota: on
               
              Volume Name:
                  gltest
              Type: Distribute
              Volume ID:
                  9e69afcc-d421-41c9-824b-7d9a31240072
              Status: Started
              Number of Bricks:
                  2
              Transport-type:
                  tcp
              Bricks:
              Brick1:
                  ikpsrv01:/gltest3/gl
              Brick2:
                  ikpsrv01:/gltest4/gl
              Options
                  Reconfigured:
              features.quota:
                  off
               
              Volume Name:
                  glusersold
              Type: Distribute
              Volume ID:
                  91a79e1a-4ff3-4191-b0ae-73357105774c
              Status: Started
              Number of Bricks:
                  1
              Transport-type:
                  tcp
              Bricks:
              Brick1:
                  ikpsrv02:/glusers2-2/gl
               
              Volume Name:
                  glusers
              Type: Distribute
              Volume ID:
                  f48b1941-9c19-4311-bd9e-497660fbcc80
              Status: Started
              Number of Bricks:
                  1
              Transport-type:
                  tcp
              Bricks:
              Brick1:
                  ikpsrv02:/glusers2-1/gl
              Options
                  Reconfigured:
              diagnostics.client-log-level:

                  CRITICAL
              diagnostics.brick-log-level:

                  CRITICAL
              features.quota:
                  off
               
              When starting
                  quotas on any of the volumes the brick will be not
                  available after some time. So we have to disable quota
                  on this volume and stop and start this volume.
               
              The command 
               
              Gluster volume
                  quota gltestn enable 
              gives the
                  following output in quotad.log
               
               
              [2015-01-07 12:01:55.522229] W
                [glusterfsd.c:1095:cleanup_and_exit]
                (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)
                [0x7fb455bd9fbd]
                (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182)
                [0x7fb455ead182]
                (-->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xd5)
                [0x7fb4569a7265]))) 0-: received signum (15), shutting
                down
              [2015-01-07
                  12:01:56.534931] I [glusterfsd.c:1959:main]
                  0-/usr/sbin/glusterfs: Started running
                  /usr/sbin/glusterfs version 3.5.2 (/usr/sbin/glusterfs
                  -s localhost --volfile-id gluster/quotad -p
                  /var/lib/glusterd/quotad/run/quotad.pid -l
                  /var/log/glusterfs/quotad.log -S
                  /var/run/fa869d5552bf340ec7506747e21b2841.socket
                  --xlator-option *replicate*.data-self-heal=off
                  --xlator-option *replicate*.metadata-self-heal=off
                  --xlator-option *replicate*.entry-self-heal=off)
              [2015-01-07
                  12:01:56.537061] I [socket.c:3561:socket_init]
                  0-socket.glusterfsd: SSL support is NOT enabled
              [2015-01-07
                  12:01:56.537134] I [socket.c:3576:socket_init]
                  0-socket.glusterfsd: using system polling thread
              [2015-01-07
                  12:01:56.537355] I [socket.c:3561:socket_init]
                  0-glusterfs: SSL support is NOT enabled
              [2015-01-07
                  12:01:56.537383] I [socket.c:3576:socket_init]
                  0-glusterfs: using system polling thread
              [2015-01-07
                  12:01:56.541480] I
                  [graph.c:254:gf_add_cmdline_options]
                  0-glwww1-replicate-0: adding option 'entry-self-heal'
                  for volume 'glwww1-replicate-0' with value 'off'
              [2015-01-07
                  12:01:56.541516] I
                  [graph.c:254:gf_add_cmdline_options]
                  0-glwww1-replicate-0: adding option
                  'metadata-self-heal' for volume 'glwww1-replicate-0'
                  with value 'off'
              [2015-01-07
                  12:01:56.541536] I
                  [graph.c:254:gf_add_cmdline_options]
                  0-glwww1-replicate-0: adding option 'data-self-heal'
                  for volume 'glwww1-replicate-0' with value 'off'
              [2015-01-07
                  12:01:56.543122] I [socket.c:3561:socket_init]
                  0-socket.quotad: SSL support is NOT enabled
              [2015-01-07
                  12:01:56.543157] I [socket.c:3576:socket_init]
                  0-socket.quotad: using system polling thread
              [2015-01-07
                  12:01:56.543404] I [dht-shared.c:311:dht_init_regex]
                  0-glwww1: using regex rsync-hash-regex =
                  ^\.(.+)\.[^.]+$
              [2015-01-07
                  12:01:56.547124] I [socket.c:3561:socket_init]
                  0-glwww1-client-1: SSL support is NOT enabled
              [2015-01-07
                  12:01:56.547158] I [socket.c:3576:socket_init]
                  0-glwww1-client-1: using system polling thread
              [2015-01-07
                  12:01:56.547728] I [socket.c:3561:socket_init]
                  0-glwww1-client-0: SSL support is NOT enabled
              [2015-01-07
                  12:01:56.547754] I [socket.c:3576:socket_init]
                  0-glwww1-client-0: using system polling thread
              [2015-01-07
                  12:01:56.547820] I [dht-shared.c:311:dht_init_regex]
                  0-gltestn: using regex rsync-hash-regex =
                  ^\.(.+)\.[^.]+$
              [2015-01-07
                  12:01:56.549471] I [socket.c:3561:socket_init]
                  0-gltestn-client-0: SSL support is NOT enabled
              [2015-01-07
                  12:01:56.549501] I [socket.c:3576:socket_init]
                  0-gltestn-client-0: using system polling thread
              [2015-01-07
                  12:01:56.549565] I [dht-shared.c:311:dht_init_regex]
                  0-gldata: using regex rsync-hash-regex =
                  ^\.(.+)\.[^.]+$
              [2015-01-07
                  12:01:56.551155] I [socket.c:3561:socket_init]
                  0-gldata-client-1: SSL support is NOT enabled
              [2015-01-07
                  12:01:56.551184] I [socket.c:3576:socket_init]
                  0-gldata-client-1: using system polling thread
              [2015-01-07
                  12:01:56.551744] I [socket.c:3561:socket_init]
                  0-gldata-client-0: SSL support is NOT enabled
              [2015-01-07
                  12:01:56.551770] I [socket.c:3576:socket_init]
                  0-gldata-client-0: using system polling thread
              [2015-01-07
                  12:01:56.551790] W
                  [graph.c:329:_log_if_unknown_option] 0-quotad: option
                  'rpc-auth.auth-glusterfs' is not recognized
              [2015-01-07
                  12:01:56.551819] W
                  [graph.c:329:_log_if_unknown_option] 0-quotad: option
                  'rpc-auth.auth-unix' is not recognized
              [2015-01-07
                  12:01:56.551838] W
                  [graph.c:329:_log_if_unknown_option] 0-quotad: option
                  'rpc-auth.auth-null' is not recognized
              [2015-01-07
                  12:01:56.551854] W
                  [graph.c:329:_log_if_unknown_option] 0-quotad: option
                  'glwww1.volume-id' is not recognized
              [2015-01-07
                  12:01:56.551870] W
                  [graph.c:329:_log_if_unknown_option] 0-quotad: option
                  'gltestn.volume-id' is not recognized
              [2015-01-07
                  12:01:56.551885] W
                  [graph.c:329:_log_if_unknown_option] 0-quotad: option
                  'gldata.volume-id' is not recognized
              [2015-01-07
                  12:01:56.551922] I [client.c:2294:notify]
                  0-glwww1-client-0: parent translators are ready,
                  attempting connect on transport
              [2015-01-07
                  12:01:56.552579] I [client.c:2294:notify]
                  0-glwww1-client-1: parent translators are ready,
                  attempting connect on transport
              [2015-01-07
                  12:01:56.553179] I [client.c:2294:notify]
                  0-gltestn-client-0: parent translators are ready,
                  attempting connect on transport
              [2015-01-07
                  12:01:56.553757] I [client.c:2294:notify]
                  0-gldata-client-0: parent translators are ready,
                  attempting connect on transport
              [2015-01-07
                  12:01:56.554372] I [client.c:2294:notify]
                  0-gldata-client-1: parent translators are ready,
                  attempting connect on transport
              Final graph:
              +------------------------------------------------------------------------------+
                1: volume
                  glwww1-client-0
                2:     type
                  protocol/client
                3:     option
                  remote-host ikpsrv03
                4:     option
                  remote-subvolume /glwwwn/gl
                5:     option
                  transport-type socket
                6:    
                option username 7c5931b4-d41e-40f7-978e-4b5a40f49481
                7:     option
                  password 6ab1661b-4752-42a4-80cc-ca0bd0167a9f
                8: end-volume
                9:
               10: volume
                  glwww1-client-1
               11:     type
                  protocol/client
               12:     option
                  remote-host ikpsrv01
               13:     option
                  remote-subvolume /glwwwn2/gl
               14:     option
                  transport-type socket
               15:    
                option username 7c5931b4-d41e-40f7-978e-4b5a40f49481
               16:     option
                  password 6ab1661b-4752-42a4-80cc-ca0bd0167a9f
               17: end-volume
               18:
               19: volume
                  glwww1-replicate-0
               20:     type
                  cluster/replicate
               21:     option
                  data-self-heal off
               22:     option
                  metadata-self-heal off
               23:     option
                  entry-self-heal off
               24:    
                  subvolumes glwww1-client-0 glwww1-client-1
               25: end-volume
               26:
               27: volume glwww1
               28:     type
                  cluster/distribute
               29:    
                  subvolumes glwww1-replicate-0
               30: end-volume
               31:
               32: volume
                  gltestn-client-0
               33:     type
                  protocol/client
               34:     option
                  remote-host ikpsrv01
               35:     option
                  remote-subvolume /gltest2/gl
               36:     option
                  transport-type socket
               37:     option
                  username 76c8bf85-a5c5-445d-8e04-e596b6eb173b
               38:     option
                  password ceec81fd-5afd-4d6b-a413-00b2688f4c62
               39: end-volume
               40:
               41: volume
                  gltestn
               42:     type
                  cluster/distribute
               43:    
                  subvolumes gltestn-client-0
               44: end-volume
               45:
               46: volume
                  gldata-client-0
               47:     type
                  protocol/client
               48:     option
                  remote-host ikpsrv02
               49:     option
                  remote-subvolume /gldata2-1/gl
               50:     option
                  transport-type socket
               51:    
                option username e9dcdbec-1110-4ab2-b667-63625126b246
               52:     option
                  password 05a9e37d-67a0-476b-88df-93d612a4e08e
               53: end-volume
               54:
               55: volume
                  gldata-client-1
               56:     type
                  protocol/client
               57:     option
                  remote-host ikpsrv01
               58:     option
                  remote-subvolume /gldata1-2/gl
               59:     option
                  transport-type socket
               60:    
                option username e9dcdbec-1110-4ab2-b667-63625126b246
               61:     option
                  password 05a9e37d-67a0-476b-88df-93d612a4e08e
               62: end-volume
               63:
               64: volume gldata
               65:     type
                  cluster/distribute
               66:    
                  subvolumes gldata-client-0 gldata-client-1
               67: end-volume
               68:
               69: volume quotad
               70:     type
                  features/quotad
               71:     option
                  rpc-auth.auth-glusterfs on
               72:     option
                  rpc-auth.auth-unix on
               73:     option
                  rpc-auth.auth-null on
               74:     option
                  transport.socket.listen-path /tmp/quotad.socket
               75:     option
                  transport-type socket
               76:     option
                  transport.address-family unix
               77:     option
                  glwww1.volume-id glwww1
               78:     option
                  gltestn.volume-id gltestn
               79:     option
                  gldata.volume-id gldata
               80:    
                  subvolumes glwww1 gltestn gldata
               81: end-volume
               82:
              +------------------------------------------------------------------------------+
              [2015-01-07
                  12:01:56.555618] I [rpc-clnt.c:1729:rpc_clnt_reconfig]
                  0-gldata-client-0: changing port to 49160 (from 0)
              [2015-01-07
                  12:01:56.556529] I [rpc-clnt.c:1729:rpc_clnt_reconfig]
                  0-glwww1-client-1: changing port to 49165 (from 0)
              [2015-01-07
                  12:01:56.556597] I [rpc-clnt.c:1729:rpc_clnt_reconfig]
                  0-gltestn-client-0: changing port to 49166 (from 0)
              [2015-01-07
                  12:01:56.556643] I [rpc-clnt.c:1729:rpc_clnt_reconfig]
                  0-gldata-client-1: changing port to 49178 (from 0)
              [2015-01-07
                  12:01:56.558382] W
                  [client-callback.c:31:client_cbk_fetchspec]
                  0-glwww1-client-0: this function should not be called
              [2015-01-07
                  12:01:56.558484] I
                  [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume
                  file changed
              [2015-01-07
                  12:01:56.558601] I
                  [client-handshake.c:1677:select_server_supported_programs]
                  0-gldata-client-0: Using Program GlusterFS 3.3, Num
                  (1298437), Version (330)
              [2015-01-07
                  12:01:56.559594] I
                  [client-handshake.c:1677:select_server_supported_programs]
                  0-gltestn-client-0: Using Program GlusterFS 3.3, Num
                  (1298437), Version (330)
              [2015-01-07
                  12:01:56.559686] I
                  [client-handshake.c:1677:select_server_supported_programs]
                  0-glwww1-client-1: Using Program GlusterFS 3.3, Num
                  (1298437), Version (330)
              [2015-01-07
                  12:01:56.559768] I
                  [client-handshake.c:1677:select_server_supported_programs]
                  0-gldata-client-1: Using Program GlusterFS 3.3, Num
                  (1298437), Version (330)
              [2015-01-07
                  12:01:56.559852] I
                  [client-handshake.c:1462:client_setvolume_cbk]
                  0-gldata-client-0: Connected to 141.52.67.86:49160,
                  attached to remote volume '/gldata2-1/gl'.
              [2015-01-07
                  12:01:56.559873] I
                  [client-handshake.c:1474:client_setvolume_cbk]
                  0-gldata-client-0: Server and Client lk-version
                  numbers are not same, reopening the fds
              [2015-01-07
                  12:01:56.559996] I
                  [client-handshake.c:1462:client_setvolume_cbk]
                  0-gltestn-client-0: Connected to 141.52.67.85:49166,
                  attached to remote volume '/gltest2/gl'.
              [2015-01-07
                  12:01:56.560017] I
                  [client-handshake.c:1474:client_setvolume_cbk]
                  0-gltestn-client-0: Server and Client lk-version
                  numbers are not same, reopening the fds
              [2015-01-07
                  12:01:56.560089] I
                  [client-handshake.c:450:client_set_lk_version_cbk]
                  0-gldata-client-0: Server lk version = 1
              [2015-01-07
                  12:01:56.560139] I
                  [client-handshake.c:1462:client_setvolume_cbk]
                  0-glwww1-client-1: Connected to 141.52.67.85:49165,
                  attached to remote volume '/glwwwn2/gl'.
              [2015-01-07
                  12:01:56.560158] I
                  [client-handshake.c:1474:client_setvolume_cbk]
                  0-glwww1-client-1: Server and Client lk-version
                  numbers are not same, reopening the fds
              [2015-01-07
                  12:01:56.560204] I [afr-common.c:4131:afr_notify]
                  0-glwww1-replicate-0: Subvolume 'glwww1-client-1' came
                  back up; going online.
              [2015-01-07
                  12:01:56.560262] I
                  [client-handshake.c:1462:client_setvolume_cbk]
                  0-gldata-client-1: Connected to 141.52.67.85:49178,
                  attached to remote volume '/gldata1-2/gl'.
              [2015-01-07
                  12:01:56.560301] I
                  [client-handshake.c:1474:client_setvolume_cbk]
                  0-gldata-client-1: Server and Client lk-version
                  numbers are not same, reopening the fds
              [2015-01-07
                  12:01:56.560385] I
                  [client-handshake.c:450:client_set_lk_version_cbk]
                  0-gltestn-client-0: Server lk version = 1
              [2015-01-07
                  12:01:56.560467] I
                  [client-handshake.c:450:client_set_lk_version_cbk]
                  0-glwww1-client-1: Server lk version = 1
              [2015-01-07
                  12:01:56.560546] I
                  [client-handshake.c:450:client_set_lk_version_cbk]
                  0-gldata-client-1: Server lk version = 1
              [2015-01-07
                  12:01:57.255911] I
                  [glusterfsd-mgmt.c:1307:mgmt_getspec_cbk] 0-glusterfs:
                  No change in volfile, continuing
              [2015-01-07
                  12:01:57.557002] I [rpc-clnt.c:1729:rpc_clnt_reconfig]
                  0-glwww1-client-0: changing port to 49153 (from 0)
              [2015-01-07
                  12:01:57.557924] I
                  [client-handshake.c:1677:select_server_supported_programs]
                  0-glwww1-client-0: Using Program GlusterFS 3.3, Num
                  (1298437), Version (330)
              [2015-01-07
                  12:01:57.558360] I
                  [client-handshake.c:1462:client_setvolume_cbk]
                  0-glwww1-client-0: Connected to 141.52.67.96:49153,
                  attached to remote volume '/glwwwn/gl'.
              [2015-01-07
                  12:01:57.558405] I
                  [client-handshake.c:1474:client_setvolume_cbk]
                  0-glwww1-client-0: Server and Client lk-version
                  numbers are not same, reopening the fds
              [2015-01-07
                  12:01:57.558706] I
                  [client-handshake.c:450:client_set_lk_version_cbk]
                  0-glwww1-client-0: Server lk version = 1
               
              what means 
               
              2015-01-07
                  12:01:56.559594] I
                  [client-handshake.c:1677:select_server_supported_programs]
                  0-gltestn-client-0: Using Program GlusterFS 3.3, Num
                  (1298437), Version (330)
               
              In the log output.
               
              Our glusterfs
                  clients all have 3.5.2 glusterfs version.
               
              Can someone help?
               
              kind regards
              Klaus

                  Bekk
               
               
              ----------------------------------------------------------
              Karlsruher

                  Institut of Technologie (KIT)
              Institut

                  für Kernphysik (IKP)
               
              Dr.

                  Klaus Bekk
               
              Hermann-von-Helmholtz-Platz

                  1
              Campus

                  Nord Gebäude 425
              76344

                  Eggenstein-Leopoldshafen
               
              Telefon:

                  +49 721 608-23382
              Fax:    

                  +49 721 608-23321
               
              E-Mail:

                  Klaus.Bekk@xxxxxxx
              http://www.kit.edu/
               
              KIT – University of the State of
                  Baden-Wuerttemberg and
              National Research Center of the Helmholtz
                  Association
              ----------------------------------------------------------
               
               
        _______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users
      
      
_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users