On 26/06/2013, at 1:42 PM, Kaushal M wrote: > Your concept seems similar to approaches 2-3 that I described, except > that its described in much more detail. The connection-group command > does to job associating peers with addresses. > But we still have the question of how do we identify peers, both > internally and externally. If we can come to an decision on this, then > I feel we have the solution to our problem. Cool. :) My thinking (so far) is that we should use the UUID of a host as its main identifier (internally). And _not_ care (from a peer point of view) which interface is in use. We could instead control the interfaces used for volumes through a defined grouping mechanism (eg like in my email, or whatever else we decide on instead). Example setup (based on my home dev lab here) is: server 1 (hostname "gluster1" atm) GlusterFS UUID = AAAAA-AAAAA-AAAAA-AAAAA-AAAAA Ethernet IP = 192.168.1.201 (DNS: gluster1) Infiniband port 0 = 172.16.101.1 (DNS: gluster1-1) Infiniband port 1 = 172.16.102.1 (DNS: gluster1-2) server 2 (hostname "gluster2" atm) GlusterFS UUID = BBBBB-BBBBB-BBBBB-BBBBB-BBBBB Ethernet IP = 192.168.1.202 (DNS: gluster2) Infiniband port 0 = 172.16.101.2 (DNS: gluster2-1) Infiniband port 1 = 172.16.102.2 (DNS: gluster2-2) server 3 (hostname "packstack" atm) GlusterFS UUID = CCCCC-CCCCC-CCCCC-CCCCC-CCCCC Ethernet IP = 192.168.1.203 (DNS: packstack) Infiniband port 0 = 172.16.101.3 (DNS: packstack-1) Infiniband port 1 = 172.16.102.3 (DNS: packstack-2) So, lets say I've just installed Gluster (next upcoming version) on them. I start glusterd on them all (first time), then peer probe each of them from each other. Shouldn't matter which address on them gets probed. When a server is probed they tell each other their (self-generated?) UUID's so they can recognise each other. (an optimisation might be for each server to also share each others list of IP addresses on their attached network cards, but that's neither here nor there) gluster1> $ sudo gluster peer probe gluster2-2 peer probe: success gluster1> $ sudo gluster peer probe packstack peer probe: success $ sudo gluster peer status Number of Peers: 2 Hostname: gluster2 Port: 24007 Uuid: BBBBB-BBBBB-BBBBB-BBBBB-BBBBB State: Peer in Cluster (Connected) Hostname: packstack Port: 24007 Uuid: CCCCC-CCCCC-CCCCC-CCCCC-CCCCC State: Peer in Cluster (Connected) For the "what should be displayed as hostname in user visible output" type of question, I can see why you're suggesting aliases. Each of my example servers has several DNS names (based on network interface IP) and it's own hostname. But, I'm not 100% sure every OS we support will reliably report back a nice "hostname" if it's asked. So, I think supporting "aliases" as a stand in for host names makes sense there. Something like: gluster1> $ sudo gluster peer alias 172.16.102.2 gluster2 peer alias: alias BBBBB-BBBBB-BBBBB-BBBBB-BBBBB to "gluster2" success gluster1> $ sudo gluster peer alias packstack1-2 packstack peer alias: alias CCCCC-CCCCC-CCCCC-CCCCC-CCCCC to "packstack" success The first argument after "alias" is a contactable gluster box, which gluster can go and fetch its UUID from. From then on, any place which needs a "hostname" for display can use the configured alias (2nd argument after "alias" above) for that server. Now, with controlling which volumes can be connected from where, I think the "connection groups" idea in my earlier email has merit, or at least is a step in the right direction. (and I'm now thinking that using "connection groups" in the peer probe command isn't the right way to go, they should just be for volumes) We could create "connection groups" for volumes as per that email, and use them to control what has access to which volumes. When something needs to be displayed, it could use the above defined alias + IP address. Something like: $ sudo gluster connection-group create client-group gluster1:192.168.1.201 gluster2:192.168.1.202 packstack:192.168.1.203 connection-group created: client-group $ sudo gluster volume groupdefault myvolume client-group volume myvolume default group: client-group I think this would give us sane ability to both use consistent naming for gluster servers, plus use multiple network interfaces intelligently. (Btw, hopefully this email isn't too long to read.) Thoughts? :) Regards and best wishes, Justin Clift -- Open Source and Standards @ Red Hat twitter.com/realjustinclift
