This is now working! I had an error in one of the config files. ><> Nathan Stratton CTO, Voila IP Communications nathan at robotics.net nathan at voilaip.com http://www.robotics.net http://www.voilaip.com On Fri, 19 Oct 2007, Nathan Allen Stratton wrote: > On Fri, 19 Oct 2007, Anand Avati wrote: > > > Nathan, > > if you have IPoIB working, using ib-verbs should be straight forward. Just > > use the IPoIB's IP addresses and transport-type as ib-verbs/{client,server} > > and things should just work. > > > > ib-sdp was a 'stopgap' solution when the ib-verbs driver was not yet > > implemented. You need ib_sdp kernel module for it to work. Also make sure > > you have 'ib_uverbs' kernel module for ib-verbs to work. If still things > > dont work, run both the server and client with -LDEBUG and attach the logs. > > Server: > > 2007-10-19 20:36:42 D [client-protocol.c:4294:client_protocol_reconnect] > brick-a-ns: attempting reconnect > 2007-10-19 20:36:42 D [ib-verbs-client.c:188:ib_verbs_client_connect] > brick-a-ns: connection on 5 success, attempting to handshake > 2007-10-19 20:36:42 E [ib-verbs.c:762:ib_verbs_handshake] > transport/ib-verbs: brick-a-ns: could not send IB handshake data > 2007-10-19 20:36:42 E [ib-verbs-client.c:197:ib_verbs_client_connect] > brick-a-ns: ib_verbs_handshake failed > 2007-10-19 20:36:42 D [client-protocol.c:4294:client_protocol_reconnect] > brick-b-ns: attempting reconnect > 2007-10-19 20:36:42 D [ib-verbs-client.c:188:ib_verbs_client_connect] > brick-b-ns: connection on 5 success, attempting to handshake > 2007-10-19 20:36:42 E [ib-verbs.c:762:ib_verbs_handshake] > transport/ib-verbs: brick-b-ns: could not send IB handshake data > 2007-10-19 20:36:42 E [ib-verbs-client.c:197:ib_verbs_client_connect] > brick-b-ns: ib_verbs_handshake failed > 2007-10-19 20:36:42 D [client-protocol.c:4294:client_protocol_reconnect] > brick-a: attempting reconnect > 2007-10-19 20:36:42 D [ib-verbs-client.c:188:ib_verbs_client_connect] > brick-a: connection on 5 success, attempting to handshake > 2007-10-19 20:36:42 E [ib-verbs.c:762:ib_verbs_handshake] > transport/ib-verbs: brick-a: could not send IB handshake data > 2007-10-19 20:36:42 E [ib-verbs-client.c:197:ib_verbs_client_connect] > brick-a: ib_verbs_handshake failed > 2007-10-19 20:36:42 D [client-protocol.c:4294:client_protocol_reconnect] > mirror-a: attempting reconnect > 2007-10-19 20:36:42 D [ib-verbs-client.c:188:ib_verbs_client_connect] > mirror-a: connection on 5 success, attempting to handshake > 2007-10-19 20:36:42 E [ib-verbs.c:762:ib_verbs_handshake] > transport/ib-verbs: mirror-a: could not send IB handshake data > 2007-10-19 20:36:42 E [ib-verbs-client.c:197:ib_verbs_client_connect] > mirror-a: ib_verbs_handshake failed > 2007-10-19 20:36:42 D [client-protocol.c:4294:client_protocol_reconnect] > brick-b: attempting reconnect > 2007-10-19 20:36:42 D [ib-verbs-client.c:188:ib_verbs_client_connect] > brick-b: connection on 5 success, attempting to handshake > 2007-10-19 20:36:42 E [ib-verbs.c:762:ib_verbs_handshake] > transport/ib-verbs: brick-b: could not send IB handshake data > 2007-10-19 20:36:42 E [ib-verbs-client.c:197:ib_verbs_client_connect] > brick-b: ib_verbs_handshake failed > 2007-10-19 20:36:42 D [client-protocol.c:4294:client_protocol_reconnect] > mirror-c: attempting reconnect > 2007-10-19 20:36:42 D [ib-verbs-client.c:188:ib_verbs_client_connect] > mirror-c: connection on 5 success, attempting to handshake > 2007-10-19 20:36:42 E [ib-verbs.c:762:ib_verbs_handshake] > transport/ib-verbs: mirror-c: could not send IB handshake data > 2007-10-19 20:36:42 E [ib-verbs-client.c:197:ib_verbs_client_connect] > mirror-c: ib_verbs_handshake failed > > Client: > > 2007-10-19 20:42:56 D [glusterfs.c:138:get_spec_fp] glusterfs: loading > spec from /usr/local/etc/glusterfs/client.vol > 2007-10-19 20:42:56 W [fuse-bridge.c:2100:fuse_transport_notify] > glusterfs-fuse: Ignoring notify event 4 > 2007-10-19 20:42:56 D [spec.y:116:new_section] libglusterfs/parser: New > node for 'share' > 2007-10-19 20:42:56 D [spec.y:132:section_type] libglusterfs/parser: > Type:share:protocol/client > 2007-10-19 20:42:56 D [xlator.c:102:xlator_set_type] libglusterfs/xlator: > attempt to load type protocol/client > 2007-10-19 20:42:56 D [xlator.c:109:xlator_set_type] libglusterfs/xlator: > attempt to load file > /usr/local/lib/glusterfs/1.3.6/xlator/protocol/client.so > 2007-10-19 20:42:56 D [spec.y:152:section_option] libglusterfs/parser: > Option:share:transport-type:ib-verbs/client > 2007-10-19 20:42:56 D [spec.y:152:section_option] libglusterfs/parser: > Option:share:remote-host:192.168.0.12 > 2007-10-19 20:42:56 D [spec.y:152:section_option] libglusterfs/parser: > Option:share:remote-subvolume:share > 2007-10-19 20:42:56 D [spec.y:216:section_end] libglusterfs/parser: > end:share > 2007-10-19 20:42:56 D [spec.y:116:new_section] libglusterfs/parser: New > node for 'writeback' > 2007-10-19 20:42:56 D [spec.y:132:section_type] libglusterfs/parser: > Type:writeback:performance/write-behind > 2007-10-19 20:42:56 D [xlator.c:102:xlator_set_type] libglusterfs/xlator: > attempt to load type performance/write-behind > 2007-10-19 20:42:56 D [xlator.c:109:xlator_set_type] libglusterfs/xlator: > attempt to load file > /usr/local/lib/glusterfs/1.3.6/xlator/performance/write-behind.so > 2007-10-19 20:42:56 W [xlator.c:156:xlator_set_type] libglusterfs/xlator: > dlsym(notify) on > /usr/local/lib/glusterfs/1.3.6/xlator/performance/write-behind.so: > undefined sym > bol: notify -- neglecting > 2007-10-19 20:42:56 D [spec.y:152:section_option] libglusterfs/parser: > Option:writeback:aggregate-size:131072 > 2007-10-19 20:42:56 D [spec.y:201:section_sub] liglusterfs/parser: > child:writeback->share > 2007-10-19 20:42:56 D [spec.y:216:section_end] libglusterfs/parser: > end:writeback > 2007-10-19 20:42:56 D [spec.y:116:new_section] libglusterfs/parser: New > node for 'readahead' > 2007-10-19 20:42:56 D [spec.y:132:section_type] libglusterfs/parser: > Type:readahead:performance/read-ahead > 2007-10-19 20:42:56 D [xlator.c:102:xlator_set_type] libglusterfs/xlator: > attempt to load type performance/read-ahead > 2007-10-19 20:42:56 D [xlator.c:109:xlator_set_type] libglusterfs/xlator: > attempt to load file > /usr/local/lib/glusterfs/1.3.6/xlator/performance/read-ahead.so > 2007-10-19 20:42:56 W [xlator.c:156:xlator_set_type] libglusterfs/xlator: > dlsym(notify) on > /usr/local/lib/glusterfs/1.3.6/xlator/performance/read-ahead.so: undefined > symbo > l: notify -- neglecting > 2007-10-19 20:42:56 D [spec.y:152:section_option] libglusterfs/parser: > Option:readahead:page-size:65536 > 2007-10-19 20:42:56 D [spec.y:152:section_option] libglusterfs/parser: > Option:readahead:page-count:16 > 2007-10-19 20:42:56 D [spec.y:201:section_sub] liglusterfs/parser: > child:readahead->writeback > 2007-10-19 20:42:56 D [spec.y:216:section_end] libglusterfs/parser: > end:readahead > 2007-10-19 20:42:56 W [inode.c:1099:inode_table_new] fuse: creating new > inode table with lru_limit=1024, sizeof(inode_t)=156 > 2007-10-19 20:42:56 D [inode.c:559:__create_inode] fuse/inode: create > inode(1) > 2007-10-19 20:42:56 D [inode.c:351:__active_inode] fuse/inode: activating > inode(1), lru=0/1024 > 2007-10-19 20:42:56 D [client-protocol.c:4549:init] share: missing > 'inode-lru-limit'. defaulting to 1000 > 2007-10-19 20:42:56 D [client-protocol.c:4566:init] share: defaulting > transport-timeout to 108 > 2007-10-19 20:42:56 D [transport.c:83:transport_load] > libglusterfs/transport: attempt to load type ib-verbs/client > 2007-10-19 20:42:56 D [transport.c:88:transport_load] > libglusterfs/transport: attempt to load file > /usr/local/lib/glusterfs/1.3.6/transport/ib-verbs/client.so > 2007-10-19 20:42:56 D [ib-verbs.c:1165:ib_verbs_options_init] > transport/ib-verbs: share: defaulting MTU to '2048' > 2007-10-19 20:42:56 D [write-behind.c:845:init] writeback: using > aggregate-size = 131072 > 2007-10-19 20:42:56 D [read-ahead.c:841:init] readahead: Using > conf->page_size = 0x10000 > 2007-10-19 20:42:56 D [read-ahead.c:848:init] readahead: Using > conf->page_count = 0x10 > 2007-10-19 20:42:56 D [client-protocol.c:4847:notify] share: got > GF_EVENT_PARENT_UP, attempting connect on transport > 2007-10-19 20:42:56 D [client-protocol.c:4294:client_protocol_reconnect] > share: attempting reconnect > 2007-10-19 20:42:56 D [ib-verbs-client.c:70:ib_verbs_client_connect] > share: socket fd = 11 > 2007-10-19 20:42:56 D [ib-verbs-client.c:88:ib_verbs_client_connect] > share: finalized on port `1023' > 2007-10-19 20:42:56 D [ib-verbs-client.c:109:ib_verbs_client_connect] > share: defaulting remote-port to 6996 > 2007-10-19 20:42:56 D [common-utils.c:161:gf_resolve_ip] resolver: DNS > cache not present, freshly probing hostname: 192.168.0.12 > 2007-10-19 20:42:56 D [common-utils.c:186:gf_resolve_ip] resolver: > returning IP:192.168.0.12[0] for hostname: 192.168.0.12 > 2007-10-19 20:42:56 D [common-utils.c:194:gf_resolve_ip] resolver: > flushing DNS cache > 2007-10-19 20:42:56 D [ib-verbs-client.c:142:ib_verbs_client_connect] > share: connect on 11 in progress (non-blocking) > 2007-10-19 20:42:56 D [ib-verbs-client.c:188:ib_verbs_client_connect] > share: connection on 11 success, attempting to handshake > 2007-10-19 20:42:56 E [ib-verbs.c:772:ib_verbs_handshake] > transport/ib-verbs: share: could not recv IB handshake-2 data > 2007-10-19 20:42:56 E [ib-verbs-client.c:197:ib_verbs_client_connect] > share: ib_verbs_handshake failed > 2007-10-19 20:42:57 D [client-protocol.c:4294:client_protocol_reconnect] > share: attempting reconnect > 2007-10-19 20:42:57 D [ib-verbs-client.c:188:ib_verbs_client_connect] > share: connection on 11 success, attempting to handshake > 2007-10-19 20:42:57 E [ib-verbs.c:762:ib_verbs_handshake] > transport/ib-verbs: share: could not send IB handshake data > 2007-10-19 20:42:57 E [ib-verbs-client.c:197:ib_verbs_client_connect] > share: ib_verbs_handshake failed > 2007-10-19 20:43:00 D [client-protocol.c:4294:client_protocol_reconnect] > share: attempting reconnect > 2007-10-19 20:43:00 D [ib-verbs-client.c:188:ib_verbs_client_connect] > share: connection on 11 success, attempting to handshake > 2007-10-19 20:43:00 E [ib-verbs.c:762:ib_verbs_handshake] > transport/ib-verbs: share: could not send IB handshake data > 2007-10-19 20:43:00 E [ib-verbs-client.c:197:ib_verbs_client_connect] > share: ib_verbs_handshake failed > 2007-10-19 20:43:03 D [client-protocol.c:4294:client_protocol_reconnect] > share: attempting reconnect > 2007-10-19 20:43:03 D [ib-verbs-client.c:188:ib_verbs_client_connect] > share: connection on 11 success, attempting to handshake > 2007-10-19 20:43:03 E [ib-verbs.c:762:ib_verbs_handshake] > transport/ib-verbs: share: could not send IB handshake data > 2007-10-19 20:43:03 E [ib-verbs-client.c:197:ib_verbs_client_connect] > share: ib_verbs_handshake failed > 2007-10-19 20:43:08 D [client-protocol.c:4294:client_protocol_reconnect] > share: attempting reconnect > 2007-10-19 20:43:08 D [ib-verbs-client.c:188:ib_verbs_client_connect] > share: connection on 11 success, attempting to handshake > 2007-10-19 20:43:08 E [ib-verbs.c:762:ib_verbs_handshake] > transport/ib-verbs: share: could not send IB handshake data > 2007-10-19 20:43:08 E [ib-verbs-client.c:197:ib_verbs_client_connect] > share: ib_verbs_handshake failed > 2007-10-19 20:43:16 D [client-protocol.c:4294:client_protocol_reconnect] > share: attempting reconnect > 2007-10-19 20:43:16 D [ib-verbs-client.c:188:ib_verbs_client_connect] > share: connection on 11 success, attempting to handshake > 2007-10-19 20:43:16 E [ib-verbs.c:762:ib_verbs_handshake] > transport/ib-verbs: share: could not send IB handshake data > 2007-10-19 20:43:16 E [ib-verbs-client.c:197:ib_verbs_client_connect] > share: ib_verbs_handshake failed > > > > _______________________________________________ > Gluster-devel mailing list > Gluster-devel@xxxxxxxxxx > http://lists.nongnu.org/mailman/listinfo/gluster-devel >
