I didn't have that exact option, but I did have
ibv-recv-work-request-count set to 32. I also added the
ib-verbs-work-request-recv-count option as well. I've tried several
values there but I still get the same error. I'm running gluter as root
on a Centos 5 box upgraded to kernel 2.6.20
I actually found it here:
https://wiki.openfabrics.org/tiki-index.php?page=Downloading+Code+From+the+OFA+git+Repositories
git link: git://git.openfabrics.org/~vlad/ofed_1_2/.git
<git://git.openfabrics.org/%7Evlad/ofed_1_2/.git>
pre-patched with the latest infiniband code from openfabrics.
below is the volume spec I'm using as well.
Thanks for your help :-)
-Mic
### File: /etc/glusterfs-server.vol - GlusterFS Server Volume Specification
volume main
type storage/posix
option directory /mnt/gluster/main
option debug off
end-volume
#volume main-io #iothreads can give performance a boost
# type performance/io-threads
# option thread-count 8
# subvolumes main
#end-volume
volume server
type protocol/server
# option transport-type tcp/server # For TCP/IP transport
# option transport-type ib-sdp/server # For Infiniband transport
option transport-type ib-verbs/server # For Infiniband Verbs transport
option ibv-recv-work-request-size 131072 # Higher performance if its
equal t o read-ahead size
option ibv-recv-work-request-count 32
option ibv-send-work-request-size 131072 # Higher performance if its
equal t o write-behind size
option ibv-send-work-request-count 32
option ib-verbs-work-request-recv-count 32
# subvolumes main-io
subvolumes main
option auth.ip.main.allow 192.168.*
end-volume
### END FILE ###
Anand Avati wrote:
Mickey,
what do you have as 'option ib-verbs-work-request-recv-count <count>'
in the server spec file? Also, are you trying to run glusterfsd as non
root? which is your distro?
thanks,
avati
2007/8/6, Mickey Mazarick <mic@xxxxxxxxxxxxxxxxxx
<mailto:mic@xxxxxxxxxxxxxxxxxx>>:
I'm having trouble getting the ibverbs transport working. I'm new
to the
infiniband world but the ib network is working correctly. I'm
using OFED
ver 1.2 but all the docs are for version 1.1.
The error in the gluster logs is:
[ib-verbs.c:1285:ib_verbs_get_device] transport/ib-verbs: server:
could
not create SRQ
Can someone point me in the right direction?
Thanks1
-Mickey Mazarick
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxx <mailto:Gluster-devel@xxxxxxxxxx>
http://lists.nongnu.org/mailman/listinfo/gluster-devel
<http://lists.nongnu.org/mailman/listinfo/gluster-devel>
--
It always takes longer than you expect, even when you take into
account Hofstadter's Law.
-- Hofstadter's Law
--
