> -----Original Message-----
> From: Jason Gunthorpe <jgg@xxxxxxxx>
> Sent: Thursday, July 26, 2018 2:04 PM
> To: Steve Wise <swise@xxxxxxxxxxxxxxxxxxxxx>
> Cc: 'Potnuri Bharat Teja' <bharat@xxxxxxxxxxx>; dledford@xxxxxxxxxx;
> rajur@xxxxxxxxxxx; linux-rdma@xxxxxxxxxxxxxxx
> Subject: Re: [PATCH for-next v2 1/2] iw_cxgb4: RDMA write with immediate
> support
>
> On Thu, Jul 26, 2018 at 12:51:06PM -0500, Steve Wise wrote:
> > > > struct {
> > > > - __be64 imm_data;
> > > > + __be32 mo;
> > > > + __be32 msn;
> > > > + /*
> > > > + * Use union for immediate data to be
consistent
> > with
> > > > + * stack's 32 bit data and iWARP spec's 64
bit data.
> > > > + */
> > > > + union {
> > > > + struct {
> > > > + u32 reserved;
> > > > + __be32 imm_data32;
> > > > + } ib_imm_data;
> > > > + __be64 imm_data64;
> > >
> > > You guys should think carefully about this as this choice to put the
> > > 32 bit version at the end becomes a permanent wire protocol ABI for
> > > iWarp.
> > >
> > > The definition of 'imm data' is a memory image of what is put into the
> > > packet, so placing the 32 bit version of this at the end of the array
> > > on the wire seems like a strange choice.
> > >
> > > If we extend verbs, it will continue to be a memory image semantic,
> > > and I would expect the 32 bits version to be the upper chunk of the
> > > array, not the lower chunk.
> > >
> >
> > Are you assuming an LE host?
>
> endianness doesn't matter, the above always orders the imm_data_32 in
> the last 4 bytes of the imm_data.
>
> > Which brings up a point that 'imm data' is not very platform
> > independent. But I guess putting it in the upper chunk is fine.
>
> imm_data is defined as a memory image. byte 0 of imm_data in the work
> request goes into bytes 0 of the packet and comes out again at byte 0
> in the work completion.
>
> This is fully platform independent.
>
> Most apps choose to put a BE value in here, but any other
> fixed-memory-order value would be fine.
My point was, the applications need to "know" what endianness their peer is,
or both agree to put it in BE or LE. IE The Verbs API doesn't do this for
them and convert from host BO to a standard protocol-defined byte order.
>
> You should also make sure your byte ordering on the wire is right, as
> this becomes ABI forever. Recommend to follow IB with byte 0 as byte 0
> in the packet. (I think, at least)
>
> Jason
Hey Bharat, please have a look at the RFC for this and see if they specify
anything.
Thanks!
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
