see below Cheers, Laurent DOUCHY. On 9/1/11 6:01 PM, Mohit Anchlia wrote: > You can check few things on 5 and 6: > > 1) gluster processes are running on node5 and 6 yes: node05 root 4902 1 0 Aug31 ? 00:00:00 /opt/glusterfs/3.2.2/sbin/glusterd root 9626 1 0 19:55 ? 00:00:00 /opt/glusterfs/3.2.2/sbin/glusterfs -f /etc/glusterd/nfs/nfs-server.vol -p /etc/glusterd/nfs/run/nfs.pid -l /var/log/glusterfs/nfs.log root 9690 9686 0 20:04 ? 00:00:00 bash -c ps -edf | grep gluster root 9704 9690 0 20:04 ? 00:00:00 grep gluster node06 root 4441 1 0 Aug31 ? 00:00:00 /opt/glusterfs/3.2.2/sbin/glusterd root 9178 1 0 19:55 ? 00:00:00 /opt/glusterfs/3.2.2/sbin/glusterfs -f /etc/glusterd/nfs/nfs-server.vol -p /etc/glusterd/nfs/run/nfs.pid -l /var/log/glusterfs/nfs.log root 9242 9238 0 20:04 ? 00:00:00 bash -c ps -edf | grep gluster root 9256 9242 0 20:04 ? 00:00:00 grep gluster > 2) both nodes are able to see each other yes: [root at node05 ~]# ping node06 PING node06.isdc.unige.ch (129.194.168.70) 56(84) bytes of data. 64 bytes from node06.isdc.unige.ch (129.194.168.70): icmp_seq=1 ttl=64 time=0.376 ms [root at node06 ~]# ping node05 PING node05.isdc.unige.ch (129.194.168.69) 56(84) bytes of data. 64 bytes from node05.isdc.unige.ch (129.194.168.69): icmp_seq=1 ttl=64 time=0.337 ms > 3) do gluster peer status on both the nodes and see what you see node 5 trust node 6 and node 6 trust node 5 [root at node05 ~]# gluster peer status Number of Peers: 5 Hostname: node08 Uuid: dd114546-5b94-4a62-9301-260703bf5707 State: Peer in Cluster (Connected) Hostname: node06 Uuid: 3142fb9a-0a6b-46ec-9262-ede95e8f798a State: Peer in Cluster (Connected) Hostname: node10 Uuid: 212ce5a0-de51-4a98-9262-ae071c2d63a0 State: Peer in Cluster (Connected) Hostname: 129.194.168.71 Uuid: a35fb0a1-af35-4a04-b38a-434f68369508 State: Peer in Cluster (Connected) Hostname: node09 Uuid: f73fee83-8d47-4f07-bfac-b8a8592eff04 State: Peer in Cluster (Connected) [root at node06 ~]# gluster peer status Number of Peers: 5 Hostname: node08 Uuid: dd114546-5b94-4a62-9301-260703bf5707 State: Peer in Cluster (Connected) Hostname: node09 Uuid: f73fee83-8d47-4f07-bfac-b8a8592eff04 State: Peer in Cluster (Connected) Hostname: node05 Uuid: 13ffcf87-6e8d-4c6b-814a-cbc14d15d88b State: Peer in Cluster (Connected) Hostname: node10 Uuid: 212ce5a0-de51-4a98-9262-ae071c2d63a0 State: Peer in Cluster (Connected) Hostname: 129.194.168.71 Uuid: a35fb0a1-af35-4a04-b38a-434f68369508 State: Peer in Cluster (Connected) > 4) check iptables same file on each node (the installation is manage by puppet) > > On Thu, Sep 1, 2011 at 8:57 AM, Laurent DOUCHY<Laurent.Douchy at unige.ch> wrote: >> It works ... >> >> [root at node07 ~]# gluster volume add-brick cluster node09:/gluster3 >> node10:/gluster3 >> Add Brick successful >> >> >> On 9/1/11 5:39 PM, Mohit Anchlia wrote: >>> Can you try with node09:/gluster3 and node10:gluster3 instead? >>> >>> On Thu, Sep 1, 2011 at 2:49 AM, Laurent DOUCHY<Laurent.Douchy at unige.ch> >>> wrote: >>>> Hi, >>>> >>>> I work on the node7 so it doesn't appear on the list. >>>> >>>> I create a folder /gluster3 on node5 and node6 and try to add them to my >>>> volume but it failed with the same message :( >>>> >>>> [root at node07 ~]# gluster volume add-brick cluster node05:/gluster3 >>>> node06:/gluster3 >>>> Operation failed on node05 >>>> >>>> next step is to reinstall from scratch the node I hope I can avoid this. >>>> >>>> On 8/31/11 9:08 PM, Mohit Anchlia wrote: >>>>> I don't see node07 in above output of gluster peer status. >>>>> >>>>> Can you try to add bricks on the hosts that gluster1, gluster2? So add >>>>> gluster3 and see if that works. >>>>> >>>>> On Wed, Aug 31, 2011 at 11:56 AM, Laurent DOUCHY >>>>> <Laurent.Douchy at unige.ch> wrote: >>>>>> Hi, >>>>>> >>>>>> I try to add 2 bricks or 4 bricks for the same effect. >>>>>> >>>>>> I try to reinstall gluster without success. >>>>>> >>>>>> Cheers, >>>>>> Laurent DOUCHY. >>>>>> >>>>>> >>>>>> On 8/31/11 8:07 PM, Burnash, James wrote: >>>>>>> Hi Laurent. >>>>>>> >>>>>>> Since your configuration specifies replication, you must add bricks in >>>>>>> the >>>>>>> same number as your number of replicas. >>>>>>> >>>>>>> For instance - if you have 2 replicas (most normal case), you would >>>>>>> need >>>>>>> to do something like this: >>>>>>> >>>>>>> gluster volume add-brick cluster node05:/gluster1 node06:/gluster1 >>>>>>> >>>>>>> James Burnash >>>>>>> Unix Engineer >>>>>>> Knight Capital Group >>>>>>> >>>>>>> >>>>>>> -----Original Message----- >>>>>>> From: gluster-users-bounces at gluster.org >>>>>>> [mailto:gluster-users-bounces at gluster.org] On Behalf Of Laurent DOUCHY >>>>>>> Sent: Wednesday, August 31, 2011 12:49 PM >>>>>>> To: gluster-users at gluster.org >>>>>>> Subject: add bricks on distributed replicated volume >>>>>>> failed >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I'm using gluster 3.2.2 on 10 nodes. Each node have 2x2 TB disk for >>>>>>> gluster. >>>>>>> >>>>>>> I manage to configure a distributed and replicated volume on 4 nodes : >>>>>>> >>>>>>> [root at node07 ~]# gluster volume info cluster >>>>>>> >>>>>>> Volume Name: cluster >>>>>>> Type: Distributed-Replicate >>>>>>> Status: Started >>>>>>> Number of Bricks: 4 x 2 = 8 >>>>>>> Transport-type: tcp >>>>>>> Bricks: >>>>>>> Brick1: node09:/gluster1 >>>>>>> Brick2: node10:/gluster1 >>>>>>> Brick3: node09:/gluster2 >>>>>>> Brick4: node10:/gluster2 >>>>>>> Brick5: node07:/gluster1 >>>>>>> Brick6: node08:/gluster1 >>>>>>> Brick7: node07:/gluster2 >>>>>>> Brick8: node08:/gluster2 >>>>>>> >>>>>>> But I can't add new nodes to this volume >>>>>>> >>>>>>> [root at node07 ~]# gluster peer status >>>>>>> Number of Peers: 5 >>>>>>> >>>>>>> Hostname: node10 >>>>>>> Uuid: 212ce5a0-de51-4a98-9262-ae071c2d63a0 >>>>>>> State: Peer in Cluster (Connected) >>>>>>> >>>>>>> Hostname: node08 >>>>>>> Uuid: dd114546-5b94-4a62-9301-260703bf5707 >>>>>>> State: Peer in Cluster (Connected) >>>>>>> >>>>>>> Hostname: node09 >>>>>>> Uuid: f73fee83-8d47-4f07-bfac-b8a8592eff04 >>>>>>> State: Peer in Cluster (Connected) >>>>>>> >>>>>>> Hostname: node06 >>>>>>> Uuid: 3142fb9a-0a6b-46ec-9262-ede95e8f798a >>>>>>> State: Peer in Cluster (Connected) >>>>>>> >>>>>>> Hostname: node05 >>>>>>> Uuid: 13ffcf87-6e8d-4c6b-814a-cbc14d15d88b >>>>>>> State: Peer in Cluster (Connected) >>>>>>> [root at node07 ~]# gluster volume add-brick cluster node05:/gluster1 >>>>>>> node06:/gluster1 node05:/gluster2 node06:/gluster2 Operation failed on >>>>>>> node05 >>>>>>> >>>>>>> I try to detach nodes 5 and 6, restart glusterd do the probe and the >>>>>>> add-brick but still nothing ... >>>>>>> >>>>>>> Did some one have any idea to fix this ? >>>>>>> >>>>>>> Thanks in advance, >>>>>>> Laurent. >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Gluster-users mailing list >>>>>>> Gluster-users at gluster.org >>>>>>> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users >>>>>>> >>>>>>> >>>>>>> DISCLAIMER: >>>>>>> This e-mail, and any attachments thereto, is intended only for use by >>>>>>> the >>>>>>> addressee(s) named herein and may contain legally privileged and/or >>>>>>> confidential information. If you are not the intended recipient of >>>>>>> this >>>>>>> e-mail, you are hereby notified that any dissemination, distribution >>>>>>> or >>>>>>> copying of this e-mail, and any attachments thereto, is strictly >>>>>>> prohibited. >>>>>>> If you have received this in error, please immediately notify me and >>>>>>> permanently delete the original and any copy of any e-mail and any >>>>>>> printout >>>>>>> thereof. E-mail transmission cannot be guaranteed to be secure or >>>>>>> error-free. The sender therefore does not accept liability for any >>>>>>> errors or >>>>>>> omissions in the contents of this message which arise as a result of >>>>>>> e-mail >>>>>>> transmission. >>>>>>> NOTICE REGARDING PRIVACY AND CONFIDENTIALITY Knight Capital Group may, >>>>>>> at >>>>>>> its discretion, monitor and review the content of all e-mail >>>>>>> communications. >>>>>>> http://www.knight.com >>>>>>> _______________________________________________ >>>>>>> Gluster-users mailing list >>>>>>> Gluster-users at gluster.org >>>>>>> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users >>>>>> _______________________________________________ >>>>>> Gluster-users mailing list >>>>>> Gluster-users at gluster.org >>>>>> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users >>>>>>
