On Wed, Feb 3, 2021 at 1:25 PM Xiao Yang <yangx.jy@xxxxxxxxxxxxxx> wrote: > > On 2021/1/29 14:28, Zhu Yanjun wrote: > > On Fri, Jan 29, 2021 at 10:16 AM Xiao Yang<yangx.jy@xxxxxxxxxxxxxx> wrote: > >> On 2021/1/28 20:54, Leon Romanovsky wrote: > >>> On Thu, Jan 28, 2021 at 02:29:43PM +0800, Xiao Yang wrote: > >>>> On 2021/1/27 20:04, Leon Romanovsky wrote: > >>>>> On Wed, Jan 27, 2021 at 04:24:31PM +0800, Xiao Yang wrote: > >>>>>> Even if we enable sq_sig_all or IBV_SEND_SIGNALED, current rxe > >>>>>> module cannot set imm_data in WC when the related WR with imm_data > >>>>>> finished on SQ. > >>>>>> > >>>>>> Signed-off-by: Xiao Yang<yangx.jy@xxxxxxxxxxxxxx> > >>>>>> --- > >>>>>> > >>>>>> Current rxe module and other rdma modules(e.g. mlx5) only set > >>>>>> imm_data in WC when the related WR with imm_data finished on RQ. > >>>>>> I am not sure if it is a expected behavior. > >>>>> This is IBTA behavior. > >>>>> > >>>>> 5.2.11 IMMEDIATE DATA EXTENDED TRANSPORT HEADER (ImmDt) - 4 BYTES > >>>>> "Immediate Data (ImmDt) contains data that is placed in the receive > >>>>> Completion Queue Element (CQE). The ImmDt is only allowed in SEND or > >>>>> RDMA WRITE packets with Immediate Data." > >>>>> > >>>>> If I understand the spec, you shouldn't set imm_data in SQ. > >>>> Hi Leon, > >>>> > >>>> About the behavior, I have another question: > >>>> For send operation with imm_data, we can verify if the delivered imm_data is > >>>> correct by CQE on RQ. > >>>> For rdma write operation with imm_data, how to verify if the delivered > >>>> imm_data is correct? :-) > >>> Probably that I didn't understand the question, but the RDMA WRITE is > >>> marked with special opcode in the BTH that indicates imm_data. > >> Hi Leon, > >> > >> The quesion is about how to get the imm_data in applications(programs in > >> user space) > > Any steps or simple method to reproduce this problem? > > I want to delve into this problem. > Hi Yanjun, > > Sorry for the wrong question. :-) > For rdma write operation with imm_data, I can get the delivered imm_data > by CQE on RQ. Wonderful. Zhu Yanjun > > Best Regards, > Xiao Yang > > Thanks a lot. > > Zhu Yanjun > > > >> 1) If client program does send operation with imm_data, server program > >> can get the delivered imm_data by calling ibv_poll_cq(&wc) > >> 2) If client program does rdma write operation with imm_data, server > >> program cannot get the delivered imm_data by calling ibv_poll_cq(&wc). > >> In this case, how does server program get the delivered imm_data? > >> > >> Best Regards, > >> Xiao Yang > >>> Thanks > >>> > >>>> Best Regards, > >>>> Xiao Yang > >>>>> Thanks > >>>>> > >>>>> > >>>>> . > >>>>> > >>>> > >>> . > >>> > >> > >> > > > > . > > > > >
