On 12/13/18 10:17 AM, Parav Pandit wrote: > > >> -----Original Message----- >> From: Mark Bloch >> Sent: Thursday, December 13, 2018 12:08 PM >> To: Parav Pandit <parav@xxxxxxxxxxxx>; Leon Romanovsky >> <leon@xxxxxxxxxx> >> Cc: Jarod Wilson <jarod@xxxxxxxxxx>; linux-rdma@xxxxxxxxxxxxxxx; Doug >> Ledford <dledford@xxxxxxxxxx>; Honggang Li <honli@xxxxxxxxxx> >> Subject: Re: [PATCH rdma-core] Add Red Hat's ibdev2netdev helper >> >> >> >> On 12/13/18 8:43 AM, Parav Pandit wrote: >>> Hi Mark, >>> >>>> -----Original Message----- >>>> From: Leon Romanovsky <leon@xxxxxxxxxx> >>>> Sent: Wednesday, December 12, 2018 1:10 AM >>>> To: Mark Bloch <markb@xxxxxxxxxxxx> >>>> Cc: Jarod Wilson <jarod@xxxxxxxxxx>; Parav Pandit >>>> <parav@xxxxxxxxxxxx>; linux-rdma@xxxxxxxxxxxxxxx; Doug Ledford >>>> <dledford@xxxxxxxxxx>; Honggang Li <honli@xxxxxxxxxx> >>>> Subject: Re: [PATCH rdma-core] Add Red Hat's ibdev2netdev helper >>>> >>>> On Wed, Dec 12, 2018 at 06:38:39AM +0000, Mark Bloch wrote: >>>>> >>>>> >>>>> On 12/11/2018 21:27, Leon Romanovsky wrote: >>>>>> On Tue, Dec 11, 2018 at 04:53:49PM -0500, Jarod Wilson wrote: >>>>>>> On 2018-12-11 4:17 PM, Parav Pandit wrote: >>>>>>>> >>>>>>>> >>>>>>>>> -----Original Message----- >>>>>>>>> From: linux-rdma-owner@xxxxxxxxxxxxxxx <linux-rdma- >>>>>>>>> owner@xxxxxxxxxxxxxxx> On Behalf Of Mark Bloch >>>>>>>>> Sent: Tuesday, December 11, 2018 1:57 PM >>>>>>>>> To: Jarod Wilson <jarod@xxxxxxxxxx>; linux-rdma@xxxxxxxxxxxxxxx >>>>>>>>> Cc: Doug Ledford <dledford@xxxxxxxxxx>; Honggang Li >>>>>>>>> <honli@xxxxxxxxxx> >>>>>>>>> Subject: Re: [PATCH rdma-core] Add Red Hat's ibdev2netdev helper >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On 12/11/18 11:46 AM, Jarod Wilson wrote: >>>>>>>>>> This is a helper script that Red Hat had previously been >>>>>>>>>> including in it's ibutils package, which is no more, with the >>>>>>>>>> retirement of ibutils and >>>>>>>>>> ibutils2 being proprietary to Mellanox now. This script still >>>>>>>>>> has use, and needs a better home, so we'd like to add it to >>>>>>>>>> rdma-core's redhat/ directory -- if not somewhere more generic, >>>>>>>>>> should other distros wish to make use of it as well. >>>>>>>>> >>>>>>>>> Why not tell users to use rdma tool (iproute2)? >>>>>>>>> >>>>>>>> Because rdma tool currently doesn’t show corresponding IPoIB >>>>>>>> upper >>>> netdevice(s) of for the IB device. >>>>>>>> For a given rdma device there can be multiple netdevices exist >>>>>>>> for >>>> RoCE ports. >>>>>>>> Rdma tool (and ibdev2netdev) doesn't show them. >>>>>>>> The scope of ibdev2netdev currently is wider than what rdma tool >>>> shows. >>>>>>>> So yes, its good recommendation to RoCE and representor users. >>>>>>> >>>>>>> We've actually had some folks notice it was missing from RHEL-8.0 >>>>>>> beta, and explicitly request it, because they have various >>>>>>> infrastructure that was expecting it to be there. It's just a >>>>>>> shell script, and could be updated along the way to use the rdma >>>>>>> tool from iproute2, or become completely obsolete at some point. >>>>>>> But we're definitely some demand to keep it around for the moment. >>>>>> >>>>>> It is already obsolete and rdmatool replaces it. I see no value in >>>>>> keeping abandoned tool in upstream repository. >>>>>> >>>>>> Regarding Parav's point, lack of IPoIB translations, iroute2 is an >>>>>> open-source project and people are encouraged to extend it. If I >>>>>> judge by number of customer requests to me (zero) to implement it, >>>>>> there is no demand in such functionality. >>>>> >>>>> Well, a quick prototype follows below, the question is should I work >>>>> on it or not, does anyone core? >>>> >>>> Parav and I are care, I have no clue about the rest of the world. >>>> >>>> It will be better to start from mockup of rdmatool output. >>>> >>>> Thanks >>> Your work is certainly useful. >>> This allows users to start using rdma tool regardless of underlying link >> layer to get ib device to net device(s) mapping. >>> >>> >> >> Ok, let's build on that, as you ( and Leon :) ) care about this, is there a format >> you prefer? >> >> What about something like this: >> rdma link -d show mlx5_3 >> /1: mlx5_3/1: subnet_prefix fe80:0000:0000:0000 lid 65535 sm_lid 0 lmc 0 >> state DOWN physical_state DISABLED >> ipoib: netdev ib0 netdev_index 42 >> ipoib: netdev ib0.3332 netdev_index 43 >> ipoib: netdev ib0.3333 netdev_index 44 >> caps: <TRAP, SL_MAP, SYS_IMAGE_GUID, CABLE_INFO, EXTENDED_SPEEDS, >> CAP_MASK2, CM, VENDOR_CLASS, CAP_MASK_NOTICE, CLIENT_REG, >> OTHER_LOCAL_CHANGES, MULT_PKER_TRAP> >> >> I assume you would like to add gid tables entries to rdma tool at some >> point, have you thought how would you like to do that? use rdma link / >> rdma gids something else? >> > > I prefer rdma addr that shows gids based on ip addresses and other ways along with their netdev. > So basically gid, and gid_attr (index, netdev, type). > Reason for addr instead of gids or link is, it aligns with ip addr. > OK cool, so no need to worry about that.. What about the output I've provided? are you okay with that? Do you want something different? I was thinking of providing the following netlink attributes: <owner:string> example: "ipoib" <ndev:string> example: "ib0" <index:u32> example: 42 Leon, as this is your baby, any thoughts? Mark
