Ok - for some reason it did not like 6 of my nodes - but able to add 34 nodes two at a time - may be client can do the similar split internally based on replica count. The failure from add-brick is simple "volume add-brick: failed: " gluster volume info Volume Name: gl_disk Type: Distributed-Replicate Volume ID: c70d525e-a255-41e2-af03-718d6dec0319 Status: Created Number of Bricks: 17 x 2 = 34 Transport-type: tcp -----Original Message----- From: gluster-users-bounces@xxxxxxxxxxx [mailto:gluster-users-bounces@xxxxxxxxxxx] On Behalf Of Prasad, Nirmal Sent: Monday, March 31, 2014 6:20 PM To: Dan Lambright Cc: gluster-users@xxxxxxxxxxx Subject: Re: 42 node gluster volume create fails silently Hi Dan, Thanks for the quick response. I'm trying to create the volume so have not reached this stage - the client exited out when I gave it : gluster volume create <vol-name> replica 2 server1:.. server2:.. .... server41:.. server42:.. If I do : gluster volume create <vol-name> replica 2 server1:.. server2:.. gluster volume add-brick <vol-name> replica 2 server3:.. server4:.. it gets me farther ... looks like there is some timeout for the gluster command - not sure - just an observation. Thanks Regards Nirmal -----Original Message----- From: Dan Lambright [mailto:dlambrig@xxxxxxxxxx] Sent: Monday, March 31, 2014 6:16 PM To: Prasad, Nirmal Cc: gluster-users@xxxxxxxxxxx Subject: Re: 42 node gluster volume create fails silently Hello, the cli logs do not contain much. If you remount your gluster volume, and create the problem again, there may be more to see. On the client side: mount -t glusterfs -o log-level=DEBUG,log-file=/tmp/my_client.log 10.16.159.219:/myvol /mnt On the server side: gluster volume set myvol diagnostics.brick-sys-log-level WARNING gluster volume set myvol diagnostics.brick-log-level WARNING You could then attach the most recent log files to your email, or the parts that seem relevant so the email is not too large. /tmp/my_client.log /var/log/glusterfs/etc*.log /var/log/glusterfs/bricks/*.log ----- Original Message ----- From: "Nirmal Prasad" <nprasad@xxxxxxxxxxx> To: gluster-users@xxxxxxxxxxx Sent: Monday, March 31, 2014 6:04:31 PM Subject: Re: 42 node gluster volume create fails silently .. and glusterd died. I had success adding individually up to 21 nodes – will go down that path. Anyone interested in log files or core files? service glusterd status glusterd dead but pid file exists From: gluster-users-bounces@xxxxxxxxxxx [mailto:gluster-users-bounces@xxxxxxxxxxx] On Behalf Of Prasad, Nirmal Sent: Monday, March 31, 2014 5:57 PM To: gluster-users@xxxxxxxxxxx Subject: Re: 42 node gluster volume create fails silently Looks symptomatic of some timeout – subsequent status command gave: gluster volume status Another transaction is in progress. Please try again after sometime. From: gluster-users-bounces@xxxxxxxxxxx [ mailto:gluster-users-bounces@xxxxxxxxxxx ] On Behalf Of Prasad, Nirmal Sent: Monday, March 31, 2014 5:53 PM To: gluster-users@xxxxxxxxxxx Subject: 42 node gluster volume create fails silently Not much of output – not sure where to see. This is the output in the cli.log – There are 42 servers and (21 brick pairs) – timeout perhaps ?? [2014-03-31 13:44:34.228467] I [cli-cmd-volume.c:1336:cli_check_gsync_present] 0-: geo-replication not installed [2014-03-31 13:44:34.229619] I [cli-cmd-volume.c:392:cli_cmd_volume_create_cbk] 0-cli: Replicate cluster type found. Checking brick order. [2014-03-31 13:44:34.230821] I [cli-cmd-volume.c:304:cli_cmd_check_brick_order] 0-cli: Brick order okay [2014-03-31 13:44:47.758977] W [rpc-transport.c:175:rpc_transport_load] 0-rpc-transport: missing 'option transport-type'. defaulting to "socket" [2014-03-31 13:44:47.763286] I [socket.c:3480:socket_init] 0-glusterfs: SSL support is NOT enabled [2014-03-31 13:44:47.763326] I [socket.c:3495:socket_init] 0-glusterfs: using system polling thread [2014-03-31 13:44:47.777000] I [cli-cmd-volume.c:1336:cli_check_gsync_present] 0-: geo-replication not installed [2014-03-31 13:44:47.780574] I [cli-rpc-ops.c:332:gf_cli_list_friends_cbk] 0-cli: Received resp to list: 0 [2014-03-31 13:44:47.782086] I [input.c:36:cli_batch] 0-: Exiting with: 0 [2014-03-31 13:46:34.231761] I [input.c:36:cli_batch] 0-: Exiting with: 110 _______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://supercolony.gluster.org/mailman/listinfo/gluster-users _______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://supercolony.gluster.org/mailman/listinfo/gluster-users _______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://supercolony.gluster.org/mailman/listinfo/gluster-users
