OK - will do - ugh. hjm On Saturday 05 November 2011 11:34:53 Dan Cyr wrote: > Try upgrading the firmware. I had this exact problem last week and > the firmware update solved it. > > Dan > > From: gluster-users-bounces at gluster.org > [mailto:gluster-users-bounces at gluster.org] On Behalf Of Harry > Mangalam Sent: Saturday, November 05, 2011 8:17 AM > To: gluster-users at gluster.org; landman at scalableinformatics.com > Subject: Re: glusterfs over rdma ... not. > > > Thanks for the help, Joe, > > This looks like the problem: > > ================================== > > $ ibv_devinfo > > Failed to get IB devices list: Function not implemented > > ================================== > > There appears to be a problem.. Either i don';t have a valid IB > driver or the device isn't recognized. lsmod says: > > $ lsmod |egrep '(ib|rdma)' > > rdma_cm 31030 0 > > ib_cm 40316 1 rdma_cm > > iw_cm 9385 1 rdma_cm > > ib_sa 22006 2 rdma_cm,ib_cm > > ib_addr 6766 1 rdma_cm > > ib_mthca 140934 0 > > ib_mad 41321 3 ib_cm,ib_sa,ib_mthca > > ib_core 64935 6 rdma_cm,ib_cm,iw_cm,ib_sa,ib_mthca,ib_mad > > so the card seems to be recognized and the driver is loaded (google > says this card is suboptimal for such an infrastructure but I > inherited this testbed so will have to deal with it for now). > > hjm > > On Friday 04 November 2011 18:32:36 Joe Landman wrote: > > On 11/04/2011 09:06 PM, Harry Mangalam wrote: > > > OK - finished some tests over tcp and ironed out a lot of > > > > > > problems. rdma is next; should be snap now.... > > > > > > > > > > > > [I must admit that this is my 1st foray into the land of IB, so > > > > > > some of the following may be obvious to a non-naive admin..] > > > > > except that while I can create and start the volume with rdma as transport: > > First things first: > > > > > > > > > > > > ibv_devinfo > > > > > > > > What does it report? > > > > > ================================== > > > > > > > > > > > > root at pbs3:~ > > > > > > > > > > > > 622 $ gluster volume info glrdma > > > > > > > > > > > > Volume Name: glrdma > > > > > > > > > > > > Type: Distribute > > > > > > > > > > > > Status: Started > > > > > > > > > > > > Number of Bricks: 4 > > > > > > > > > > > > Transport-type: tcp,rdma > > > > > > > > > > > > Bricks: > > > > > > > > > > > > Brick1: pbs1:/data2 > > > > > > > > > > > > Brick2: pbs2:/data2 > > > > > > > > > > > > Brick3: pbs3:/data2 > > > > > > > > > > > > Brick4: pbs3:/data > > > > > > > > > > > > ================================== > > > > > > > > > > > > I can't mount the damn thing. This seems to be a fairly > > > frequent > > > > > > problem according to google.. Again, all servers and clients > > > are > > > > > > ubuntu 10.04.3/64b, running self-compiled 3.3b1. > > > > try a > > > > > > > > showmount -e pbs2 > > > > > > > > Also, did you enable the auth bits as last time? > > -- > > Harry Mangalam - Research Computing, OIT, Rm 225 MSTB, UC Irvine > > [ZOT 2225] / 92697 Google Voice Multiplexer: (949) 478-4487 > > MSTB Lat/Long: (33.642025,-117.844414) (paste into Google Maps) > > -- > > This signature has been OCCUPIED! -- Harry Mangalam - Research Computing, OIT, Rm 225 MSTB, UC Irvine [ZOT 2225] / 92697 Google Voice Multiplexer: (949) 478-4487 MSTB Lat/Long: (33.642025,-117.844414) (paste into Google Maps) -- This signature has been OCCUPIED! -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://gluster.org/pipermail/gluster-users/attachments/20111105/8a51d0dc/attachment-0001.htm>
