On Mon, Sep 17, 2018 at 11:01:59AM -0600, Jason Gunthorpe wrote:
> On Mon, Sep 17, 2018 at 10:22:04AM -0500, Steve Wise wrote:
> > > > >>>> Why? This is not current practice for rdma devices: the core names
> > > them.
> > > > >>>
> > > > >>> That is a bad practice.. Leon has been working on device renaming
> > like
> > > > >>> netdev has, so it would make no sense to have no name here and have
> > > > >>> renaming down the road.
> > > > >>>
> > > > >>> Jason
> > > > >>>
> > > > >>
> > > > >> This causes a slit kink in arbitrarily naming rxe rdma devices. From
> > > > >> providers/rxe/rxe.c:
> > > > >>
> > > > >> static const struct verbs_match_ent hca_table[] = {
> > > > >> /* FIXME: rxe needs a more reliable way to detect the rxe
> > device */
> > > > >> VERBS_NAME_MATCH("rxe", NULL),
> > > > >> {},
> > > > >> };
> > > > >>
> > > > >>
> > > > >> So when I add a new rxe device named "foo0", it gets added fine in
> > the
> > > > >> kernel, but the user space side skips it. It doesn't show up in
> > > > >> ibv_devices or ibv_devinfo for example. But the sysfs entries are
> > > there...
> > > > >
> > > > > Yes, this needs fixing too :(
> > > > >
> > > > > I was hoping we'd get to netlink discovery before this became an
> > > > > issue.. The right solution is to use the new driver id number, I
> > > > > think.
> > > > >
> > > > > Jason
> > > > >
> > > >
> > > > Do you mean the rdma_driver_id enum? That would I methinks...
> > >
> > > Neltink ibdev index + rdma_driver_id will give us perfect match of
> > > underlying device.
> > >
> > > Thanks
> >
> > So currently there is this verbs_match_ent struct and VERBS_MATCH* macros in
> > libibverbs/driver.h. These allow providers to specify either a PCI
> > vendor/device ID or a string that the libibverbs core uses to match
> > providers to devices. The string is a hack to support thinks like RXE that
> > don't have a specific vendor/device ID since it supports any arbitrary NIC
> > device.
> >
> > Should verbs_match_ent and friends be enhanced, for example, to add match by
> > driver_id?
>
> Probably, yes, unless there is some other way we can avoid this in the
> short term..
>
> > And currently the kernel side does nothing with driver_id.
>
> The ioctl path checks it.
>
> > Should it export the driver id to sysfs?
>
> I was hoping we could discover this over netlink and avoid changing
> sysfs.. But timelines might not work out for that now - Leon?
I was thinking on this some more, and we don't really need to wait for
netlink or to add uverbs stuff to netlink.
We should add a new ioctl to return the 'driver description' from
uverbs.
This changes the rdma-core flow a bit as it cannot bind the driver
until after opening the uverbs FD, but it sure makes more sense ..
Jason
