> -----Original Message-----
> From: oulijun <oulijun@xxxxxxxxxx>
> Sent: Wednesday, February 27, 2019 9:28 PM
> To: Parav Pandit <parav@xxxxxxxxxxxx>; bvanassche@xxxxxxx; linux-
> rdma@xxxxxxxxxxxxxxx
> Subject: Re: [EXPERIMENTAL v1 0/4] RDMA loopback device
>
> 在 2019/2/28 5:32, Parav Pandit 写道:
> >
> >> -----Original Message-----
> >> From: oulijun <oulijun@xxxxxxxxxx>
> >> Sent: Wednesday, February 27, 2019 2:09 AM
> >> To: Parav Pandit <parav@xxxxxxxxxxxx>; bvanassche@xxxxxxx; linux-
> >> rdma@xxxxxxxxxxxxxxx
> >> Subject: Re: [EXPERIMENTAL v1 0/4] RDMA loopback device
> >>
> >> Hi,
> >> What is the difference between loopback driver and loopback packet
> >> in the IB protocol?
> >>
> > Loopback packet in IB protocol is as you described.
> > However users almost always needs a physical hardware HCA (IB/RoCE,
> iwarp).
> > Iwarp patches are in progress.
> >
> > There hasn't been reliable software stack yet for users who want to just
> run their applications in laptop/vm on single system.
> > This is often done for unit tests, sanity, stack regression tests.
> >
> > Loopback driver mimics the netdev's lo style loopback rdma device in very
> simple manner.
> > Its ABI less, where there almost any user space driver at all and still
> achieves 7Gbps to 80Gbps performance.
> >
> > Again rxe and cma can be enhanced, to add lo netdev interface via new
> netlink command added.
> > But I have serious doubts that we can make rxe really work perfectly
> without ABI change, and without major rework.
> > I am not sure if that is really worth it. If siw reworks the rxe driver, than
> there is some light of hope.
> >
> Thank your detail reply。 I have seen the goal for loopback driver。
> But I still want to use this series to ask about the loopback packet problem.
>
> The protocol specifies:
> o10-6.2-1.1: If the CI supports the Loopback Indicator, the CI shall report it
> through the Query HCA Verb. The CI shall accept the Loopback Indicator as
> an input modifier to the Create Address Handle, Modify Address Handle,
> Modify QP and Modify EEC verbs
>
> I have checked the create address handle verbs' input modifier.
> struct rdma_ah_attr {
> struct ib_global_route grh;
> u8 sl;
> u8 static_rate;
> u8 port_num;
> u8 ah_flags;
> enum rdma_ah_attr_type type;
> union {
> struct ib_ah_attr ib;
> struct roce_ah_attr roce;
> struct opa_ah_attr opa;
> };
> };
>
> and the verb for create ah in protocol specification.
> input modifiers:
> ...
> ...
> • Loopback Indicator (if the HCA supports Loopback Indicator on Address
> Handles). DLID and Loopback Indicator are mutually exclusive.
> ...
>
> I don't see why defined the loopback indicator and expose for app?
> I understand that the loopback indicator should be gived by ofed, the user
> can configue it. Any other understand?
> if the user not use lo nedev for loopback and use other nedev for rdma
> device?
>
Loopack still works where HCA loops this packets for RoCE and IB link layers without this user configuration.
A user explicit flag would be required when user wants to force loopback when SLID != DLID.
I do know the use case or supported HCAs which does that.
> thanks
>
> >> for the IB protocol description:
> >> 10.2.2.3 LOOPBACK
> >> Packet loopback is supported through self addressed packets as
> >> described in
> >> 17.2.2 and C17-18:. Self-addressed packet is a packet whose DLID and
> >> SLID address the same port of the same CA. Additionally, an HCA can
> >> optionally support a Loopback Indicator. When Loopback Indicator is
> >> supported, both Address Handles and QP/EE Address Vectors can use
> >> Loopback Indicator instead of Destination LID. When reaping
> >> completions for Datagrams based QPs, the Loopback Indicator is also
> >> reported if the origin of the message was Loopback.
> >>
> >> I always have a question for the above. Why not expsure the loopback
> >> indicator for user to support loopback packet?
> >>
> > It is not useful to expose this and modify all user applications for hw and
> sw devices.
> > Core code already does this detection and does the loopback on hw and
> sw interfaces.
> > Hence it is not desired to expose this optional part of the spec.
>
>
