On 1/16/12 11:02 PM, Amar Tumballi wrote:
>
>> [2012-01-16 19:17:41.946893] E [rdma.c:198:rdma_new_post]
>> 0-rpc-transport/rdma: memory registration failed
>> [2012-01-16 19:17:41.946932] E [rdma.c:2341:__rdma_create_posts]
>> 0-rpc-transport/rdma: rdma.gtest-server: post creation failed
>> [2012-01-16 19:17:41.948827] E [rdma.c:3861:rdma_get_device]
>> 0-rpc-transport/rdma: rdma.gtest-server: could not allocate posts
>
> What is the memory available on the machine?
>
>
I have confirmed this on two different sets of hosts (Sun X4170 and a
Dell Precision WorkStation R5400). Both sets of hosts are on the same
IB subnet with Qlogic hardware. This is all running on RHEL5.
Sun X4170
# free -m
total used free shared buffers cached
Mem: 24083 19643 4440 0 407 18524
-/+ buffers/cache: 711 23372
Swap: 2047 0 2047
Dell Precision WorkStation R5400
# free -m
total used free shared buffers cached
Mem: 16041 4512 11529 0 623 3409
-/+ buffers/cache: 479 15562
Swap: 2047 0 2047
# ib_write_bw -a 192.168.1.101
------------------------------------------------------------------
RDMA_Write BW Test
Number of qp's running 1
Connection type : RC
Each Qp will post up to 100 messages each time
Inline data is used up to 0 bytes message
local address: LID 0x09, QPN 0x000d, PSN 0x9362ce RKey 0x40a80b00
VAddr 0x002b07f6b1a000
remote address: LID 0x20, QPN 0x000c, PSN 0x6ef146, RKey 0x40900a00
VAddr 0x002aeb16efe000
Mtu : 2048
------------------------------------------------------------------
#bytes #iterations BW peak[MB/sec] BW average[MB/sec]
2 5000 0.37 0.37
4 5000 0.74 0.74
8 5000 1.48 1.48
16 5000 2.98 2.97
32 5000 5.91 5.91
64 5000 11.75 11.74
128 5000 23.47 23.47
256 5000 45.79 45.79
512 5000 90.65 90.63
1024 5000 175.00 174.96
2048 5000 322.49 320.08
4096 5000 419.93 419.46
8192 5000 478.97 478.96
16384 5000 511.44 505.41
32768 5000 516.93 516.67
65536 5000 517.94 517.50
131072 5000 519.76 517.61
262144 5000 520.55 517.76
524288 5000 518.28 517.91
1048576 5000 521.42 518.09
2097152 5000 607.73 527.41
4194304 5000 623.20 606.25
8388608 5000 617.73 607.03
------------------------------------------------------------------
--
---
Derek T. Yarnell
University of Maryland
Institute for Advanced Computer Studies
