On Friday, July 07/27/18, 2018 at 00:46:00 +0530, Steve Wise wrote:
>
>
> > -----Original Message-----
> > From: Jason Gunthorpe <jgg@xxxxxxxx>
> > Sent: Thursday, July 26, 2018 2:04 PM
> > To: Steve Wise <swise@xxxxxxxxxxxxxxxxxxxxx>
> > Cc: 'Potnuri Bharat Teja' <bharat@xxxxxxxxxxx>; dledford@xxxxxxxxxx;
> > rajur@xxxxxxxxxxx; linux-rdma@xxxxxxxxxxxxxxx
> > Subject: Re: [PATCH for-next v2 1/2] iw_cxgb4: RDMA write with immediate
> > support
> >
> > On Thu, Jul 26, 2018 at 12:51:06PM -0500, Steve Wise wrote:
> > > > > struct {
> > > > > - __be64 imm_data;
> > > > > + __be32 mo;
> > > > > + __be32 msn;
> > > > > + /*
> > > > > + * Use union for immediate data to be
> consistent
> > > with
> > > > > + * stack's 32 bit data and iWARP spec's 64
> bit data.
> > > > > + */
> > > > > + union {
> > > > > + struct {
> > > > > + u32 reserved;
> > > > > + __be32 imm_data32;
> > > > > + } ib_imm_data;
> > > > > + __be64 imm_data64;
> > > >
> > > > You guys should think carefully about this as this choice to put the
> > > > 32 bit version at the end becomes a permanent wire protocol ABI for
> > > > iWarp.
> > > >
> > > > The definition of 'imm data' is a memory image of what is put into the
> > > > packet, so placing the 32 bit version of this at the end of the array
> > > > on the wire seems like a strange choice.
> > > >
> > > > If we extend verbs, it will continue to be a memory image semantic,
> > > > and I would expect the 32 bits version to be the upper chunk of the
> > > > array, not the lower chunk.
> > > >
> > >
> > > Are you assuming an LE host?
> >
> > endianness doesn't matter, the above always orders the imm_data_32 in
> > the last 4 bytes of the imm_data.
> >
> > > Which brings up a point that 'imm data' is not very platform
> > > independent. But I guess putting it in the upper chunk is fine.
> >
> > imm_data is defined as a memory image. byte 0 of imm_data in the work
> > request goes into bytes 0 of the packet and comes out again at byte 0
> > in the work completion.
> >
> > This is fully platform independent.
> >
> > Most apps choose to put a BE value in here, but any other
> > fixed-memory-order value would be fine.
>
> My point was, the applications need to "know" what endianness their peer is,
> or both agree to put it in BE or LE. IE The Verbs API doesn't do this for
> them and convert from host BO to a standard protocol-defined byte order.
>
> >
> > You should also make sure your byte ordering on the wire is right, as
> > this becomes ABI forever. Recommend to follow IB with byte 0 as byte 0
> > in the packet. (I think, at least)
> >
> > Jason
>
> Hey Bharat, please have a look at the RFC for this and see if they specify
> anything.
>
Spec has nothing much specified about the Byte Order or any details as such.
https://tools.ietf.org/html/rfc7306#section-6
Thanks,
Bharat.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
