On Mon, Sep 24, 2018 at 02:20:43PM -0600, Jason Gunthorpe wrote:
> On Mon, Sep 17, 2018 at 11:01:59AM -0600, Jason Gunthorpe wrote:
> > On Mon, Sep 17, 2018 at 10:22:04AM -0500, Steve Wise wrote:
> > > > > >>>> Why? This is not current practice for rdma devices: the core names
> > > > them.
> > > > > >>>
> > > > > >>> That is a bad practice.. Leon has been working on device renaming
> > > like
> > > > > >>> netdev has, so it would make no sense to have no name here and have
> > > > > >>> renaming down the road.
> > > > > >>>
> > > > > >>> Jason
> > > > > >>>
> > > > > >>
> > > > > >> This causes a slit kink in arbitrarily naming rxe rdma devices. From
> > > > > >> providers/rxe/rxe.c:
> > > > > >>
> > > > > >> static const struct verbs_match_ent hca_table[] = {
> > > > > >> /* FIXME: rxe needs a more reliable way to detect the rxe
> > > device */
> > > > > >> VERBS_NAME_MATCH("rxe", NULL),
> > > > > >> {},
> > > > > >> };
> > > > > >>
> > > > > >>
> > > > > >> So when I add a new rxe device named "foo0", it gets added fine in
> > > the
> > > > > >> kernel, but the user space side skips it. It doesn't show up in
> > > > > >> ibv_devices or ibv_devinfo for example. But the sysfs entries are
> > > > there...
> > > > > >
> > > > > > Yes, this needs fixing too :(
> > > > > >
> > > > > > I was hoping we'd get to netlink discovery before this became an
> > > > > > issue.. The right solution is to use the new driver id number, I
> > > > > > think.
> > > > > >
> > > > > > Jason
> > > > > >
> > > > >
> > > > > Do you mean the rdma_driver_id enum? That would I methinks...
> > > >
> > > > Neltink ibdev index + rdma_driver_id will give us perfect match of
> > > > underlying device.
> > > >
> > > > Thanks
> > >
> > > So currently there is this verbs_match_ent struct and VERBS_MATCH* macros in
> > > libibverbs/driver.h. These allow providers to specify either a PCI
> > > vendor/device ID or a string that the libibverbs core uses to match
> > > providers to devices. The string is a hack to support thinks like RXE that
> > > don't have a specific vendor/device ID since it supports any arbitrary NIC
> > > device.
> > >
> > > Should verbs_match_ent and friends be enhanced, for example, to add match by
> > > driver_id?
> >
> > Probably, yes, unless there is some other way we can avoid this in the
> > short term..
> >
> > > And currently the kernel side does nothing with driver_id.
> >
> > The ioctl path checks it.
> >
> > > Should it export the driver id to sysfs?
> >
> > I was hoping we could discover this over netlink and avoid changing
> > sysfs.. But timelines might not work out for that now - Leon?
>
> I was thinking on this some more, and we don't really need to wait for
> netlink or to add uverbs stuff to netlink.
>
> We should add a new ioctl to return the 'driver description' from
> uverbs.
>
> This changes the rdma-core flow a bit as it cannot bind the driver
> until after opening the uverbs FD, but it sure makes more sense ..
It may be helpful for rdma-core, but for Steve's work it won't. He is
not supposed to touch uverbs file just to understand on which software
RDMA device to operate. It means that knowledge of driver_id will be
exposed over netlink anyway, and I think that it won't be cleanest thing
to duplicate interfaces.
Thanks
>
> Jason
Attachment:
signature.asc
Description: PGP signature
