Re: Fwd: to mailing list gluster.org

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Vijaikumar, Sachin will work on this quota issue.

Pranith
On 01/22/2015 12:46 AM, Pranith Kumar Karampuri wrote:
+ quota devs

Pranith
On 01/07/2015 07:37 PM, Bekk, Klaus (IKP) wrote:

 

Hi all,

 

we are running 4 server with Ubuntu 14.04 as glusterfs server with 3.5.2-ubuntu1~trusty1.


When starting quotas on any of the volumes the brick will be not available after some time. So we have to disable quota on this volume and stop and start this volume.

 

 

We have the following configuration

 

root@ikpsrv01:/var/log/glusterfs# gluster volume info

 

Volume Name: gltestn

Type: Distribute

Volume ID: e50bd30d-dbc8-4279-ac9d-2e557328a643

Status: Started

Number of Bricks: 1

Transport-type: tcp

Bricks:

Brick1: ikpsrv01:/gltest2/gl

 

Volume Name: glwww1

Type: Replicate

Volume ID: 811e4c2a-4520-4e61-bb84-955afbdb9dff

Status: Started

Number of Bricks: 1 x 2 = 2

Transport-type: tcp

Bricks:

Brick1: ikpsrv03:/glwwwn/gl

Brick2: ikpsrv01:/glwwwn2/gl

Options Reconfigured:

features.quota: on

 

Volume Name: corsika

Type: Distribute

Volume ID: d6e07f3e-4b35-4736-9b71-afc3224f0f29

Status: Started

Number of Bricks: 4

Transport-type: tcp

Bricks:

Brick1: ikpsrv03:/glcors3-1/gl

Brick2: ikpsrv03:/glcors3-2/gl

Brick3: ikpsrv02:/glcors2-1/gl

Brick4: ikpsrv02:/glcors2-2/gl

 

Volume Name: gldata

Type: Distribute

Volume ID: 82485d03-9ef4-4151-a2cc-881072e871a1

Status: Started

Number of Bricks: 2

Transport-type: tcp

Bricks:

Brick1: ikpsrv02:/gldata2-1/gl

Brick2: ikpsrv01:/gldata1-2/gl

Options Reconfigured:

features.quota: on

 

Volume Name: gltest

Type: Distribute

Volume ID: 9e69afcc-d421-41c9-824b-7d9a31240072

Status: Started

Number of Bricks: 2

Transport-type: tcp

Bricks:

Brick1: ikpsrv01:/gltest3/gl

Brick2: ikpsrv01:/gltest4/gl

Options Reconfigured:

features.quota: off

 

Volume Name: glusersold

Type: Distribute

Volume ID: 91a79e1a-4ff3-4191-b0ae-73357105774c

Status: Started

Number of Bricks: 1

Transport-type: tcp

Bricks:

Brick1: ikpsrv02:/glusers2-2/gl

 

Volume Name: glusers

Type: Distribute

Volume ID: f48b1941-9c19-4311-bd9e-497660fbcc80

Status: Started

Number of Bricks: 1

Transport-type: tcp

Bricks:

Brick1: ikpsrv02:/glusers2-1/gl

Options Reconfigured:

diagnostics.client-log-level: CRITICAL

diagnostics.brick-log-level: CRITICAL

features.quota: off

 

When starting quotas on any of the volumes the brick will be not available after some time. So we have to disable quota on this volume and stop and start this volume.

 

The command

 

Gluster volume quota gltestn enable

gives the following output in quotad.log

 

 

 

 

[2015-01-07 12:01:55.522229] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fb455bd9fbd] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fb455ead182] (-->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xd5) [0x7fb4569a7265]))) 0-: received signum (15), shutting down

[2015-01-07 12:01:56.534931] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.5.2 (/usr/sbin/glusterfs -s localhost --volfile-id gluster/quotad -p /var/lib/glusterd/quotad/run/quotad.pid -l /var/log/glusterfs/quotad.log -S /var/run/fa869d5552bf340ec7506747e21b2841.socket --xlator-option *replicate*.data-self-heal=off --xlator-option *replicate*.metadata-self-heal=off --xlator-option *replicate*.entry-self-heal=off)

[2015-01-07 12:01:56.537061] I [socket.c:3561:socket_init] 0-socket.glusterfsd: SSL support is NOT enabled

[2015-01-07 12:01:56.537134] I [socket.c:3576:socket_init] 0-socket.glusterfsd: using system polling thread

[2015-01-07 12:01:56.537355] I [socket.c:3561:socket_init] 0-glusterfs: SSL support is NOT enabled

[2015-01-07 12:01:56.537383] I [socket.c:3576:socket_init] 0-glusterfs: using system polling thread

[2015-01-07 12:01:56.541480] I [graph.c:254:gf_add_cmdline_options] 0-glwww1-replicate-0: adding option 'entry-self-heal' for volume 'glwww1-replicate-0' with value 'off'

[2015-01-07 12:01:56.541516] I [graph.c:254:gf_add_cmdline_options] 0-glwww1-replicate-0: adding option 'metadata-self-heal' for volume 'glwww1-replicate-0' with value 'off'

[2015-01-07 12:01:56.541536] I [graph.c:254:gf_add_cmdline_options] 0-glwww1-replicate-0: adding option 'data-self-heal' for volume 'glwww1-replicate-0' with value 'off'

[2015-01-07 12:01:56.543122] I [socket.c:3561:socket_init] 0-socket.quotad: SSL support is NOT enabled

[2015-01-07 12:01:56.543157] I [socket.c:3576:socket_init] 0-socket.quotad: using system polling thread

[2015-01-07 12:01:56.543404] I [dht-shared.c:311:dht_init_regex] 0-glwww1: using regex rsync-hash-regex = ^\.(.+)\.[^.]+$

[2015-01-07 12:01:56.547124] I [socket.c:3561:socket_init] 0-glwww1-client-1: SSL support is NOT enabled

[2015-01-07 12:01:56.547158] I [socket.c:3576:socket_init] 0-glwww1-client-1: using system polling thread

[2015-01-07 12:01:56.547728] I [socket.c:3561:socket_init] 0-glwww1-client-0: SSL support is NOT enabled

[2015-01-07 12:01:56.547754] I [socket.c:3576:socket_init] 0-glwww1-client-0: using system polling thread

[2015-01-07 12:01:56.547820] I [dht-shared.c:311:dht_init_regex] 0-gltestn: using regex rsync-hash-regex = ^\.(.+)\.[^.]+$

[2015-01-07 12:01:56.549471] I [socket.c:3561:socket_init] 0-gltestn-client-0: SSL support is NOT enabled

[2015-01-07 12:01:56.549501] I [socket.c:3576:socket_init] 0-gltestn-client-0: using system polling thread

[2015-01-07 12:01:56.549565] I [dht-shared.c:311:dht_init_regex] 0-gldata: using regex rsync-hash-regex = ^\.(.+)\.[^.]+$

[2015-01-07 12:01:56.551155] I [socket.c:3561:socket_init] 0-gldata-client-1: SSL support is NOT enabled

[2015-01-07 12:01:56.551184] I [socket.c:3576:socket_init] 0-gldata-client-1: using system polling thread

[2015-01-07 12:01:56.551744] I [socket.c:3561:socket_init] 0-gldata-client-0: SSL support is NOT enabled

[2015-01-07 12:01:56.551770] I [socket.c:3576:socket_init] 0-gldata-client-0: using system polling thread

[2015-01-07 12:01:56.551790] W [graph.c:329:_log_if_unknown_option] 0-quotad: option 'rpc-auth.auth-glusterfs' is not recognized

[2015-01-07 12:01:56.551819] W [graph.c:329:_log_if_unknown_option] 0-quotad: option 'rpc-auth.auth-unix' is not recognized

[2015-01-07 12:01:56.551838] W [graph.c:329:_log_if_unknown_option] 0-quotad: option 'rpc-auth.auth-null' is not recognized

[2015-01-07 12:01:56.551854] W [graph.c:329:_log_if_unknown_option] 0-quotad: option 'glwww1.volume-id' is not recognized

[2015-01-07 12:01:56.551870] W [graph.c:329:_log_if_unknown_option] 0-quotad: option 'gltestn.volume-id' is not recognized

[2015-01-07 12:01:56.551885] W [graph.c:329:_log_if_unknown_option] 0-quotad: option 'gldata.volume-id' is not recognized

[2015-01-07 12:01:56.551922] I [client.c:2294:notify] 0-glwww1-client-0: parent translators are ready, attempting connect on transport

[2015-01-07 12:01:56.552579] I [client.c:2294:notify] 0-glwww1-client-1: parent translators are ready, attempting connect on transport

[2015-01-07 12:01:56.553179] I [client.c:2294:notify] 0-gltestn-client-0: parent translators are ready, attempting connect on transport

[2015-01-07 12:01:56.553757] I [client.c:2294:notify] 0-gldata-client-0: parent translators are ready, attempting connect on transport

[2015-01-07 12:01:56.554372] I [client.c:2294:notify] 0-gldata-client-1: parent translators are ready, attempting connect on transport

Final graph:

+------------------------------------------------------------------------------+

  1: volume glwww1-client-0

  2:     type protocol/client

  3:     option remote-host ikpsrv03

  4:     option remote-subvolume /glwwwn/gl

  5:     option transport-type socket

  6:     option username 7c5931b4-d41e-40f7-978e-4b5a40f49481

  7:     option password 6ab1661b-4752-42a4-80cc-ca0bd0167a9f

  8: end-volume

  9:

10: volume glwww1-client-1

11:     type protocol/client

12:     option remote-host ikpsrv01

13:     option remote-subvolume /glwwwn2/gl

14:     option transport-type socket

15:     option username 7c5931b4-d41e-40f7-978e-4b5a40f49481

16:     option password 6ab1661b-4752-42a4-80cc-ca0bd0167a9f

17: end-volume

18:

19: volume glwww1-replicate-0

20:     type cluster/replicate

21:     option data-self-heal off

22:     option metadata-self-heal off

23:     option entry-self-heal off

24:     subvolumes glwww1-client-0 glwww1-client-1

25: end-volume

26:

27: volume glwww1

28:     type cluster/distribute

29:     subvolumes glwww1-replicate-0

30: end-volume

31:

32: volume gltestn-client-0

33:     type protocol/client

34:     option remote-host ikpsrv01

35:     option remote-subvolume /gltest2/gl

36:     option transport-type socket

37:     option username 76c8bf85-a5c5-445d-8e04-e596b6eb173b

38:     option password ceec81fd-5afd-4d6b-a413-00b2688f4c62

39: end-volume

40:

41: volume gltestn

42:     type cluster/distribute

43:     subvolumes gltestn-client-0

44: end-volume

45:

46: volume gldata-client-0

47:     type protocol/client

48:     option remote-host ikpsrv02

49:     option remote-subvolume /gldata2-1/gl

50:     option transport-type socket

51:     option username e9dcdbec-1110-4ab2-b667-63625126b246

52:     option password 05a9e37d-67a0-476b-88df-93d612a4e08e

53: end-volume

54:

55: volume gldata-client-1

56:     type protocol/client

57:     option remote-host ikpsrv01

58:     option remote-subvolume /gldata1-2/gl

59:     option transport-type socket

60:     option username e9dcdbec-1110-4ab2-b667-63625126b246

61:     option password 05a9e37d-67a0-476b-88df-93d612a4e08e

62: end-volume

63:

64: volume gldata

65:     type cluster/distribute

66:     subvolumes gldata-client-0 gldata-client-1

67: end-volume

68:

69: volume quotad

70:     type features/quotad

71:     option rpc-auth.auth-glusterfs on

72:     option rpc-auth.auth-unix on

73:     option rpc-auth.auth-null on

74:     option transport.socket.listen-path /tmp/quotad.socket

75:     option transport-type socket

76:     option transport.address-family unix

77:     option glwww1.volume-id glwww1

78:     option gltestn.volume-id gltestn

79:     option gldata.volume-id gldata

80:     subvolumes glwww1 gltestn gldata

81: end-volume

82:

+------------------------------------------------------------------------------+

[2015-01-07 12:01:56.555618] I [rpc-clnt.c:1729:rpc_clnt_reconfig] 0-gldata-client-0: changing port to 49160 (from 0)

[2015-01-07 12:01:56.556529] I [rpc-clnt.c:1729:rpc_clnt_reconfig] 0-glwww1-client-1: changing port to 49165 (from 0)

[2015-01-07 12:01:56.556597] I [rpc-clnt.c:1729:rpc_clnt_reconfig] 0-gltestn-client-0: changing port to 49166 (from 0)

[2015-01-07 12:01:56.556643] I [rpc-clnt.c:1729:rpc_clnt_reconfig] 0-gldata-client-1: changing port to 49178 (from 0)

[2015-01-07 12:01:56.558382] W [client-callback.c:31:client_cbk_fetchspec] 0-glwww1-client-0: this function should not be called

[2015-01-07 12:01:56.558484] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed

[2015-01-07 12:01:56.558601] I [client-handshake.c:1677:select_server_supported_programs] 0-gldata-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330)

[2015-01-07 12:01:56.559594] I [client-handshake.c:1677:select_server_supported_programs] 0-gltestn-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330)

[2015-01-07 12:01:56.559686] I [client-handshake.c:1677:select_server_supported_programs] 0-glwww1-client-1: Using Program GlusterFS 3.3, Num (1298437), Version (330)

[2015-01-07 12:01:56.559768] I [client-handshake.c:1677:select_server_supported_programs] 0-gldata-client-1: Using Program GlusterFS 3.3, Num (1298437), Version (330)

[2015-01-07 12:01:56.559852] I [client-handshake.c:1462:client_setvolume_cbk] 0-gldata-client-0: Connected to 141.52.67.86:49160, attached to remote volume '/gldata2-1/gl'.

[2015-01-07 12:01:56.559873] I [client-handshake.c:1474:client_setvolume_cbk] 0-gldata-client-0: Server and Client lk-version numbers are not same, reopening the fds

[2015-01-07 12:01:56.559996] I [client-handshake.c:1462:client_setvolume_cbk] 0-gltestn-client-0: Connected to 141.52.67.85:49166, attached to remote volume '/gltest2/gl'.

[2015-01-07 12:01:56.560017] I [client-handshake.c:1474:client_setvolume_cbk] 0-gltestn-client-0: Server and Client lk-version numbers are not same, reopening the fds

[2015-01-07 12:01:56.560089] I [client-handshake.c:450:client_set_lk_version_cbk] 0-gldata-client-0: Server lk version = 1

[2015-01-07 12:01:56.560139] I [client-handshake.c:1462:client_setvolume_cbk] 0-glwww1-client-1: Connected to 141.52.67.85:49165, attached to remote volume '/glwwwn2/gl'.

[2015-01-07 12:01:56.560158] I [client-handshake.c:1474:client_setvolume_cbk] 0-glwww1-client-1: Server and Client lk-version numbers are not same, reopening the fds

[2015-01-07 12:01:56.560204] I [afr-common.c:4131:afr_notify] 0-glwww1-replicate-0: Subvolume 'glwww1-client-1' came back up; going online.

[2015-01-07 12:01:56.560262] I [client-handshake.c:1462:client_setvolume_cbk] 0-gldata-client-1: Connected to 141.52.67.85:49178, attached to remote volume '/gldata1-2/gl'.

[2015-01-07 12:01:56.560301] I [client-handshake.c:1474:client_setvolume_cbk] 0-gldata-client-1: Server and Client lk-version numbers are not same, reopening the fds

[2015-01-07 12:01:56.560385] I [client-handshake.c:450:client_set_lk_version_cbk] 0-gltestn-client-0: Server lk version = 1

[2015-01-07 12:01:56.560467] I [client-handshake.c:450:client_set_lk_version_cbk] 0-glwww1-client-1: Server lk version = 1

[2015-01-07 12:01:56.560546] I [client-handshake.c:450:client_set_lk_version_cbk] 0-gldata-client-1: Server lk version = 1

[2015-01-07 12:01:57.255911] I [glusterfsd-mgmt.c:1307:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing

[2015-01-07 12:01:57.557002] I [rpc-clnt.c:1729:rpc_clnt_reconfig] 0-glwww1-client-0: changing port to 49153 (from 0)

[2015-01-07 12:01:57.557924] I [client-handshake.c:1677:select_server_supported_programs] 0-glwww1-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330)

[2015-01-07 12:01:57.558360] I [client-handshake.c:1462:client_setvolume_cbk] 0-glwww1-client-0: Connected to 141.52.67.96:49153, attached to remote volume '/glwwwn/gl'.

[2015-01-07 12:01:57.558405] I [client-handshake.c:1474:client_setvolume_cbk] 0-glwww1-client-0: Server and Client lk-version numbers are not same, reopening the fds

[2015-01-07 12:01:57.558706] I [client-handshake.c:450:client_set_lk_version_cbk] 0-glwww1-client-0: Server lk version = 1

 

what means

 

2015-01-07 12:01:56.559594] I [client-handshake.c:1677:select_server_supported_programs] 0-gltestn-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330)

 

In the log output.

 

Our glusterfs clients all have 3.5.2 glusterfs version.

 

Can someone help?

 

kind regards

Klaus Bekk

 

 

 

----------------------------------------------------------

Karlsruher Institut of Technologie (KIT)

Institut für Kernphysik (IKP)

 

Dr. Klaus Bekk

 

Hermann-von-Helmholtz-Platz 1

Campus Nord Gebäude 425

76344 Eggenstein-Leopoldshafen

 

Telefon: +49 721 608-23382

Fax:     +49 721 608-23321

 

E-Mail: Klaus.Bekk@xxxxxxx

http://www.kit.edu/

 

KIT – University of the State of Baden-Wuerttemberg and

National Research Center of the Helmholtz Association

----------------------------------------------------------

 



_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users


_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users

[Index of Archives]     [Gluster Development]     [Linux Filesytems Development]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Bugtraq]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux