On Mon, Sep 17, 2018 at 12:38:47PM -0500, Steve Wise wrote:
>
>
> > -----Original Message-----
> > From: linux-rdma-owner@xxxxxxxxxxxxxxx <linux-rdma-
> > owner@xxxxxxxxxxxxxxx> On Behalf Of Jason Gunthorpe
> > Sent: Monday, September 17, 2018 12:02 PM
> > To: Steve Wise <swise@xxxxxxxxxxxxxxxxxxxxx>
> > Cc: 'Leon Romanovsky' <leon@xxxxxxxxxx>; 'Bernard Metzler'
> > <BMT@xxxxxxxxxxxxxx>; linux-rdma@xxxxxxxxxxxxxxx
> > Subject: Re: [PATCH v3 00/13] Request for Comments on SoftiWarp
> >
> > On Mon, Sep 17, 2018 at 10:22:04AM -0500, Steve Wise wrote:
> > > > > >>>> Why? This is not current practice for rdma devices: the core
> > names
> > > > them.
> > > > > >>>
> > > > > >>> That is a bad practice.. Leon has been working on device
> renaming
> > > like
> > > > > >>> netdev has, so it would make no sense to have no name here and
> > have
> > > > > >>> renaming down the road.
> > > > > >>>
> > > > > >>> Jason
> > > > > >>>
> > > > > >>
> > > > > >> This causes a slit kink in arbitrarily naming rxe rdma devices.
> From
> > > > > >> providers/rxe/rxe.c:
> > > > > >>
> > > > > >> static const struct verbs_match_ent hca_table[] = {
> > > > > >> /* FIXME: rxe needs a more reliable way to detect the rxe
> > > device */
> > > > > >> VERBS_NAME_MATCH("rxe", NULL),
> > > > > >> {},
> > > > > >> };
> > > > > >>
> > > > > >>
> > > > > >> So when I add a new rxe device named "foo0", it gets added fine
> in
> > > the
> > > > > >> kernel, but the user space side skips it. It doesn't show up in
> > > > > >> ibv_devices or ibv_devinfo for example. But the sysfs entries
> are
> > > > there...
> > > > > >
> > > > > > Yes, this needs fixing too :(
> > > > > >
> > > > > > I was hoping we'd get to netlink discovery before this became an
> > > > > > issue.. The right solution is to use the new driver id number, I
> > > > > > think.
> > > > > >
> > > > > > Jason
> > > > > >
> > > > >
> > > > > Do you mean the rdma_driver_id enum? That would I methinks...
> > > >
> > > > Neltink ibdev index + rdma_driver_id will give us perfect match of
> > > > underlying device.
> > > >
> > > > Thanks
> > >
> > > So currently there is this verbs_match_ent struct and VERBS_MATCH*
> > macros in
> > > libibverbs/driver.h. These allow providers to specify either a PCI
> > > vendor/device ID or a string that the libibverbs core uses to match
> > > providers to devices. The string is a hack to support thinks like RXE
> that
> > > don't have a specific vendor/device ID since it supports any arbitrary
> NIC
> > > device.
> > >
> > > Should verbs_match_ent and friends be enhanced, for example, to add
> > match by
> > > driver_id?
> >
> > Probably, yes, unless there is some other way we can avoid this in the
> > short term..
> >
>
> Currently librxe is telling the core to match any rdma device names that
> begin with "rxe". Can we change that to have librxe tell the core to match
> any rdma devices that come from module "rdma_rxe"? Its just as ugly, but
> would work until we finalize the driver_id design.
>
>
> > > And currently the kernel side does nothing with driver_id.
> >
> > The ioctl path checks it.
> >
> > > Should it export the driver id to sysfs?
> >
> > I was hoping we could discover this over netlink and avoid changing
> > sysfs.. But timelines might not work out for that now - Leon?
> >
>
> So the goal is to get rid of sysfs as the way libibverbs gets info on kernel
> side objects? and instead use netlink?
At least for auto discovery, yes
>
> Steve.
>
Attachment:
signature.asc
Description: PGP signature
