See the comment at the top of the file. Posting the SEND really isn't
"needed", because the addr/rkey/len doesn't change. But the wire
protocol allows the two buffers to be different, so the client posts the
SEND each time to tell the server where to RDMA the payload to/from...
Steve.
On 2/25/2019 1:53 PM, Garegin Grigoryan (RIT Student) wrote:
> Thanks!
>
> Sorry for the wrong link.
> Here's the code I am asking about in the question number 3. What is
> the difference between these two parts and why do we need to post to
> the send queue twice?
> static int rping_test_client(struct rping_cb *cb){
> ........
> rping_format_send(cb, cb->start_buf, cb->start_mr);
> ret = ibv_post_send(cb->qp, &cb->sq_wr, &bad_wr);
> .......
> rping_format_send(cb, cb->rdma_buf, cb->rdma_mr);
> ret = ibv_post_send(cb->qp, &cb->sq_wr, &bad_wr);
> .......
> }
>
> On Mon, Feb 25, 2019 at 2:41 PM Steve Wise <swise@xxxxxxxxxxxxxxxxxxxxx> wrote:
>> Hey,
>>
>> On 2/25/2019 11:05 AM, Garegin Grigoryan (RIT Student) wrote:
>>> Hi,
>>>
>>> I'm new to RDMA and trying to understand rping.c
>>> (https://github.com/ofiwg/librdmacm/blob/master/examples/rping.c)
>>> code.
>> You should be looking at the latest rdma-core repo at:
>>
>> https://github.com/linux-rdma/rdma-core
>>
>>
>>> I have three questions so far.
>>> 1. What is the fundamental difference between cb->rdma_buf and
>>> cb->start_buf? Why rdma_buf is needed at all?
>>
>> Nothing other than the "ping source" data originates, at the rping
>> client, from start_buf and the "pingponged" data lands in rdma_buf at
>> the client. It might help to think of rping as a "test tool" that
>> implements a traditional "icmp ping" over and RDMA connection. And, it
>> is designed to utilize all the main rdma operations: SEND, RECV, READ,
>> and WRITE operations. So it is not optimized in any way other than to
>> exercise these features of rdma.
>>
>>
>>> 2. Does rping_format_send just print out the message to-be-sent?
>> It adds the buffer virtual address, the rkey, and length to the SEND
>> message that is sent to the server for this rping "iteration". The
>> server uses that address, rkey, and length to post and RDMA READ to pull
>> the "ping" data from the client to the server. The fact that these
>> values don't change between iterations irrelevant. The rping wire
>> protocol assumes the client sends this information to the server for
>> every "ping" iteration.
>>
>> Note this comment at the front of rping.c:
>>
>> /*
>> * rping "ping/pong" loop:
>> * client sends source rkey/addr/len
>> * server receives source rkey/add/len
>> * server rdma reads "ping" data from source
>> * server sends "go ahead" on rdma read completion
>> * client sends sink rkey/addr/len
>> * server receives sink rkey/addr/len
>> * server rdma writes "pong" data to sink
>> * server sends "go ahead" on rdma write completion
>> * <repeat loop>
>> */
>>
>>
>>> 3. Line 980 is same as 964. Does it post the same message to the send
>>> queue? I can not find if there have been any changes to cb->sq_wr
>>> between 964 and 980.
>>
>> I'm not sure which lines you are referring to. Pleaes cite the latest
>> rdma-core repo code, and include the src code snipit to ask specific
>> questions...
>>
>>
>>> Thank you.
>>> --
>>> Sincerely,
>>> Garegin Grigoryan
>
>
