Looks to me that the second glusterfsd is not getting connected to the client. This is the output of the first glusterfsd, from startup to completion of the running of gclusterfs - it shows the client connecting: [root@daisy gluster]# glusterfsd -f /home/jad/gluster/server0.vol -l /dev/stdout -L DEBUG -N [Mar 08 18:37:49] [DEBUG/spec.y:113/new_section()] libglusterfs/parser:new_section: New node for 'brick' [Mar 08 18:37:49] [DEBUG/spec.y:129/section_type()] libglusterfs/parser:section_type: Type:brick:storage/posix [Mar 08 18:37:49] [DEBUG/xlator.c:93/xlator_set_type()] libglusterfs/xlator:xlator_set_type: attempt to load type storage/posix [Mar 08 18:37:49] [DEBUG/xlator.c:100/xlator_set_type()] libglusterfs/xlator: xlator_set_type: attempt to load file /usr/local/lib/glusterfs/1.3.0-pre2/xlator/storage/posix.so [Mar 08 18:37:49] [DEBUG/spec.y:149/section_option()] libglusterfs/parser:section_option: Option:brick:directory:/home/jad/gluster/cfs0 [Mar 08 18:37:49] [DEBUG/spec.y:206/section_end()] libglusterfs/parser:section_end: end:brick [Mar 08 18:37:49] [DEBUG/spec.y:113/new_section()] libglusterfs/parser:new_section: New node for 'server' [Mar 08 18:37:49] [DEBUG/spec.y:129/section_type()] libglusterfs/parser:section_type: Type:server:protocol/server [Mar 08 18:37:49] [DEBUG/xlator.c:93/xlator_set_type()] libglusterfs/xlator:xlator_set_type: attempt to load type protocol/server [Mar 08 18:37:49] [DEBUG/xlator.c:100/xlator_set_type()] libglusterfs/xlator: xlator_set_type: attempt to load file /usr/local/lib/glusterfs/1.3.0-pre2/xlator/protocol/server.so [Mar 08 18:37:49] [DEBUG/spec.y:149/section_option()] libglusterfs/parser:section_option: Option:server:transport-type:tcp/server [Mar 08 18:37:49] [DEBUG/spec.y:149/section_option()] libglusterfs/parser:section_option: Option:server:bind-address:127.0.0.1 [Mar 08 18:37:49] [DEBUG/spec.y:149/section_option()] libglusterfs/parser:section_option: Option:server:listen-port:6996 [Mar 08 18:37:49] [DEBUG/spec.y:191/section_sub()] liglusterfs/parser:section_sub: child:server->brick [Mar 08 18:37:49] [DEBUG/spec.y:149/section_option()] libglusterfs/parser:section_option: Option:server:auth.ip.brick.allow:* [Mar 08 18:37:49] [DEBUG/spec.y:206/section_end()] libglusterfs/parser:section_end: end:server [Mar 08 18:37:49] [DEBUG/proto-srv.c:2918/init()] protocol/server:protocol/server xlator loaded [Mar 08 18:37:49] [DEBUG/transport.c:83/transport_load()] libglusterfs/transport:transport_load: attempt to load type tcp/server [Mar 08 18:37:49] [DEBUG/transport.c:88/transport_load()] libglusterfs/transport:transport_load: attempt to load file /usr/local/lib/glusterfs/1.3.0-pre2/transport/tcp/server.so [Mar 08 18:38:34] [DEBUG/tcp-server.c:148/tcp_server_notify()] tcp/server:Registering socket (4) for new transport object of 127.0.0.1 [Mar 08 18:38:34] [DEBUG/proto-srv.c:2419/mop_setvolume()] server-protocol:mop_setvolume: received port = 1023 [Mar 08 18:38:34] [DEBUG/proto-srv.c:2435/mop_setvolume()] server-protocol:mop_setvolume: IP addr = *, received ip addr = 127.0.0.1 [Mar 08 18:38:34] [DEBUG/proto-srv.c:2445/mop_setvolume()] server-protocol:mop_setvolume: accepted client from 127.0.0.1 [Mar 08 18:38:34] [DEBUG/tcp-server.c:148/tcp_server_notify()] tcp/server:Registering socket (7) for new transport object of 127.0.0.1 [Mar 08 18:38:34] [DEBUG/proto-srv.c:2419/mop_setvolume()] server-protocol:mop_setvolume: received port = 1022 [Mar 08 18:38:34] [DEBUG/proto-srv.c:2435/mop_setvolume()] server-protocol:mop_setvolume: IP addr = *, received ip addr = 127.0.0.1 [Mar 08 18:38:34] [DEBUG/proto-srv.c:2445/mop_setvolume()] server-protocol:mop_setvolume: accepted client from 127.0.0.1 This is the output of the second glusterdfs - no sign of the client connecting: [root@daisy gluster]# glusterfsd -f /home/jad/gluster/server1.vol -l /dev/stdout -L DEBUG -N [Mar 08 18:38:02] [DEBUG/spec.y:113/new_section()] libglusterfs/parser:new_section: New node for 'brick' [Mar 08 18:38:02] [DEBUG/spec.y:129/section_type()] libglusterfs/parser:section_type: Type:brick:storage/posix [Mar 08 18:38:02] [DEBUG/xlator.c:93/xlator_set_type()] libglusterfs/xlator:xlator_set_type: attempt to load type storage/posix [Mar 08 18:38:02] [DEBUG/xlator.c:100/xlator_set_type()] libglusterfs/xlator: xlator_set_type: attempt to load file /usr/local/lib/glusterfs/1.3.0-pre2/xlator/storage/posix.so [Mar 08 18:38:02] [DEBUG/spec.y:149/section_option()] libglusterfs/parser:section_option: Option:brick:directory:/home/jad/gluster/cfs1 [Mar 08 18:38:02] [DEBUG/spec.y:206/section_end()] libglusterfs/parser:section_end: end:brick [Mar 08 18:38:02] [DEBUG/spec.y:113/new_section()] libglusterfs/parser:new_section: New node for 'server2' [Mar 08 18:38:02] [DEBUG/spec.y:129/section_type()] libglusterfs/parser:section_type: Type:server2:protocol/server [Mar 08 18:38:02] [DEBUG/xlator.c:93/xlator_set_type()] libglusterfs/xlator:xlator_set_type: attempt to load type protocol/server [Mar 08 18:38:02] [DEBUG/xlator.c:100/xlator_set_type()] libglusterfs/xlator: xlator_set_type: attempt to load file /usr/local/lib/glusterfs/1.3.0-pre2/xlator/protocol/server.so [Mar 08 18:38:02] [DEBUG/spec.y:149/section_option()] libglusterfs/parser:section_option: Option:server2:transport-type:tcp/server [Mar 08 18:38:02] [DEBUG/spec.y:149/section_option()] libglusterfs/parser:section_option: Option:server2:bind-address:127.0.0.1 [Mar 08 18:38:02] [DEBUG/spec.y:149/section_option()] libglusterfs/parser:section_option: Option:server2:listen-port:6997 [Mar 08 18:38:02] [DEBUG/spec.y:191/section_sub()] liglusterfs/parser:section_sub: child:server2->brick [Mar 08 18:38:02] [DEBUG/spec.y:149/section_option()] libglusterfs/parser:section_option: Option:server2:auth.ip.brick.allow:* [Mar 08 18:38:02] [DEBUG/spec.y:206/section_end()] libglusterfs/parser:section_end: end:server2 [Mar 08 18:38:02] [DEBUG/proto-srv.c:2918/init()] protocol/server:protocol/server xlator loaded [Mar 08 18:38:02] [DEBUG/transport.c:83/transport_load()] libglusterfs/transport:transport_load: attempt to load type tcp/server [Mar 08 18:38:02] [DEBUG/transport.c:88/transport_load()] libglusterfs/transport:transport_load: attempt to load file /usr/local/lib/glusterfs/1.3.0-pre2/transport/tcp/server.so And for completeness, the output of running glusterdfs: [root@daisy gluster]# glusterfs -f /home/jad/gluster/client.vol -l /dev/stdout -L DEBUG -N fs [Mar 08 18:38:34] [DEBUG/glusterfs.c:196/get_spec_fp()] glusterfs:loading spec from /home/jad/gluster/client.vol [Mar 08 18:38:34] [DEBUG/spec.y:113/new_section()] libglusterfs/parser:new_section: New node for 'brick' [Mar 08 18:38:34] [DEBUG/spec.y:129/section_type()] libglusterfs/parser:section_type: Type:brick:protocol/client [Mar 08 18:38:34] [DEBUG/xlator.c:93/xlator_set_type()] libglusterfs/xlator:xlator_set_type: attempt to load type protocol/client [Mar 08 18:38:34] [DEBUG/xlator.c:100/xlator_set_type()] libglusterfs/xlator: xlator_set_type: attempt to load file /usr/local/lib/glusterfs/1.3.0-pre2/xlator/protocol/client.so [Mar 08 18:38:34] [DEBUG/spec.y:149/section_option()] libglusterfs/parser:section_option: Option:brick:transport-type:tcp/client [Mar 08 18:38:34] [DEBUG/spec.y:149/section_option()] libglusterfs/parser:section_option: Option:brick:remote-host:127.0.0.1 [Mar 08 18:38:34] [DEBUG/spec.y:149/section_option()] libglusterfs/parser:section_option: Option:brick:remote-port:6996 [Mar 08 18:38:34] [DEBUG/spec.y:149/section_option()] libglusterfs/parser:section_option: Option:brick:remote-subvolume:brick [Mar 08 18:38:34] [DEBUG/spec.y:206/section_end()] libglusterfs/parser:section_end: end:brick [Mar 08 18:38:34] [DEBUG/spec.y:113/new_section()] libglusterfs/parser:new_section: New node for 'brick-afr' [Mar 08 18:38:34] [DEBUG/spec.y:129/section_type()] libglusterfs/parser:section_type: Type:brick-afr:protocol/client [Mar 08 18:38:34] [DEBUG/xlator.c:93/xlator_set_type()] libglusterfs/xlator:xlator_set_type: attempt to load type protocol/client [Mar 08 18:38:34] [DEBUG/xlator.c:100/xlator_set_type()] libglusterfs/xlator: xlator_set_type: attempt to load file /usr/local/lib/glusterfs/1.3.0-pre2/xlator/protocol/client.so [Mar 08 18:38:34] [DEBUG/spec.y:149/section_option()] libglusterfs/parser:section_option: Option:brick-afr:transport-type:tcp/client [Mar 08 18:38:34] [DEBUG/spec.y:149/section_option()] libglusterfs/parser:section_option: Option:brick-afr:remote-host:127.0.0.1 [Mar 08 18:38:34] [DEBUG/spec.y:149/section_option()] libglusterfs/parser:section_option: Option:brick-afr:report-port:6997 [Mar 08 18:38:34] [DEBUG/spec.y:149/section_option()] libglusterfs/parser:section_option: Option:brick-afr:remote-subvolume:brick [Mar 08 18:38:34] [DEBUG/spec.y:206/section_end()] libglusterfs/parser:section_end: end:brick-afr [Mar 08 18:38:34] [DEBUG/spec.y:113/new_section()] libglusterfs/parser:new_section: New node for 'afr' [Mar 08 18:38:34] [DEBUG/spec.y:129/section_type()] libglusterfs/parser:section_type: Type:afr:cluster/afr [Mar 08 18:38:34] [DEBUG/xlator.c:93/xlator_set_type()] libglusterfs/xlator:xlator_set_type: attempt to load type cluster/afr [Mar 08 18:38:34] [DEBUG/xlator.c:100/xlator_set_type()] libglusterfs/xlator: xlator_set_type: attempt to load file /usr/local/lib/glusterfs/1.3.0-pre2/xlator/cluster/afr.so [Mar 08 18:38:34] [DEBUG/spec.y:191/section_sub()] liglusterfs/parser:section_sub: child:afr->brick [Mar 08 18:38:34] [DEBUG/spec.y:191/section_sub()] liglusterfs/parser:section_sub: child:afr->brick-afr [Mar 08 18:38:34] [DEBUG/spec.y:149/section_option()] libglusterfs/parser:section_option: Option:afr:replicate:*:2 [Mar 08 18:38:34] [DEBUG/spec.y:206/section_end()] libglusterfs/parser:section_end: end:afr [Mar 08 18:38:34] [DEBUG/afr.c:2148/init()] afr:xlator name is brick [Mar 08 18:38:34] [DEBUG/afr.c:2148/init()] afr:xlator name is brick-afr [Mar 08 18:38:34] [DEBUG/afr.c:2152/init()] afr:child count 2 [Mar 08 18:38:34] [DEBUG/afr.c:2168/init()] afr:afr->init: afr server not specified, defaulting to brick [Mar 08 18:38:34] [DEBUG/transport.c:83/transport_load()] libglusterfs/transport:transport_load: attempt to load type tcp/client [Mar 08 18:38:34] [DEBUG/transport.c:88/transport_load()] libglusterfs/transport:transport_load: attempt to load file /usr/local/lib/glusterfs/1.3.0-pre2/transport/tcp/client.so [Mar 08 18:38:34] [DEBUG/tcp-client.c:174/tcp_connect()] transport: tcp: :try_connect: socket fd = 7 [Mar 08 18:38:34] [DEBUG/tcp-client.c:196/tcp_connect()] transport: tcp: :try_connect: finalized on port `1023' [Mar 08 18:38:34] [DEBUG/tcp-client.c:220/tcp_connect()] tcp/client:try_connect: defaulting remote-port to 6996 [Mar 08 18:38:34] [DEBUG/tcp-client.c:255/tcp_connect()] tcp/client:connect on 7 in progress (non-blocking) [Mar 08 18:38:34] [DEBUG/tcp-client.c:301/tcp_connect()] tcp/client:connection on 7 success, attempting to handshake [Mar 08 18:38:34] [DEBUG/tcp-client.c:58/do_handshake()] transport/tcp-client:dictionary length = 50 [Mar 08 18:38:34] [DEBUG/transport.c:83/transport_load()] libglusterfs/transport:transport_load: attempt to load type tcp/client [Mar 08 18:38:34] [DEBUG/transport.c:88/transport_load()] libglusterfs/transport:transport_load: attempt to load file /usr/local/lib/glusterfs/1.3.0-pre2/transport/tcp/client.so [Mar 08 18:38:34] [DEBUG/tcp-client.c:174/tcp_connect()] transport: tcp: :try_connect: socket fd = 8 [Mar 08 18:38:34] [DEBUG/tcp-client.c:196/tcp_connect()] transport: tcp: :try_connect: finalized on port `1022' [Mar 08 18:38:34] [DEBUG/tcp-client.c:255/tcp_connect()] tcp/client:connect on 8 in progress (non-blocking) [Mar 08 18:38:34] [DEBUG/tcp-client.c:301/tcp_connect()] tcp/client:connection on 8 success, attempting to handshake [Mar 08 18:38:34] [DEBUG/tcp-client.c:58/do_handshake()] transport/tcp-client:dictionary length = 50 So the question now seems to be: Why is the client not connecting to the second glusterdfsd??? Thanks, j On Thu, 8 Mar 2007, Anand Avati wrote: > > James, > Jut to make sure the second glusterfsd (of cfs1) is connected to the > client, can you attach the log file of glusterfsd for cfs1? > > avati > > On Thu, Mar 08, 2007 at 01:58:26PM +0000, James Dyer wrote: > > I've been trying for ages to get afr to work, but somethings either > > broken, or I'm just not 'getting it'... > > > > I'm running glusterfs 1.3.0-pre2 on RHEL 4 > > > > At the moment, I'm just trying to get two instances of glusterdfsd running > > on the same server, with the directories /home/jad/gluster/cfs0 and cfs1 > > respectively. If I understand things correctly (and it's quite possible > > I'm not understanding properly), the following configuration should mean > > that any file I create in the glusterfs directory should appear in both > > cfs0 and cfs1, due to the replicate option. > > > > Configuration for first glusterdfs: > > volume brick > > type storage/posix > > option directory /home/jad/gluster/cfs0 > > end-volume > > > > volume server > > type protocol/server > > option transport-type tcp/server > > option bind-address 127.0.0.1 > > option listen-port 6996 > > subvolumes brick > > option auth.ip.brick.allow * > > end-volume > > > > Configuration for second glusterdfs: > > volume brick > > type storage/posix > > option directory /home/jad/gluster/cfs1 > > end-volume > > > > volume server > > type protocol/server > > option transport-type tcp/server > > option bind-address 127.0.0.1 > > option listen-port 6997 > > subvolumes brick > > option auth.ip.brick.allow * > > end-volume > > > > Configuration for glusterfs mounting: > > volume brick > > type protocol/client > > option transport-type tcp/client > > option remote-host 127.0.0.1 > > option remote-port 6996 > > option remote-subvolume brick > > end-volume > > > > volume brick-afr > > type protocol/client > > option transport-type tcp/client > > option remote-host 127.0.0.1 > > option report-port 6997 > > option remote-subvolume brick > > end-volume > > > > volume afr > > type cluster/afr > > subvolumes brick brick-afr > > option replicate *:2 > > end-volume > > > > > > > > First instance of glusterdfs run with: > > glusterfsd -f /home/jad/gluster/server0.vol > > > > Second instance: > > glusterfsd -f /home/jad/gluster/server1.vol > > > > glusterfs mount run with: > > glusterfs -f ./client.vol glusterfs > > > > > > > > What I'm finding however, is that if I do a 'touch glusterfs/foo.bar', the > > file is appearing in cfs0, but not in cfs1, which seems to me means that > > gluster is ignoring my 'option replicate *:2' entry in the afr volume. > > > > > > Any thoughts on what I'm doing wrong??? > > > > James > > > > > > > > _______________________________________________ > > Gluster-devel mailing list > > Gluster-devel@xxxxxxxxxx > > http://lists.nongnu.org/mailman/listinfo/gluster-devel > > > >
