On Fri, Feb 7, 2025 at 10:36 AM Jakub Kicinski <kuba@xxxxxxxxxx> wrote: > > On Thu, 6 Feb 2025 22:17:58 -0500 Andy Gospodarek wrote: > > On Thu, Feb 6, 2025 at 7:44 PM Jakub Kicinski <kuba@xxxxxxxxxx> wrote: > > > On Thu, 6 Feb 2025 20:13:32 -0400 Jason Gunthorpe wrote: > > > > From: Andy Gospodarek <gospo@xxxxxxxxxxxx> > > > > > > > > Signed-off-by: Andy Gospodarek <gospo@xxxxxxxxxxxx> > > > > Signed-off-by: Jason Gunthorpe <jgg@xxxxxxxxxx> > > > > > > This is only needed for RDMA, why can't you make this part of bnxt_re ? > > > > This is not just needed for RDMA, so having the aux device for fwctl > > as part of the base driver is preferred. > > Please elaborate. As you well know I have experience using Broadcom > devices in large TCP/IP networks, without the need for proprietary > tooling. I totally get that. As a user it is not satisfying to have to download and attempt to compile complicated proprietary tools to use hardware features that seem like they should just work. I don't think fwctl should be used as a crutch to avoid doing the work that is needed to get support upstream. > Now, I understand that it may be expedient for Broadcom and nVidia > to skip the upstream process and ship "features" to customers using > DOCA and whatever you call your tooling. But let's be honest that > this is the motivation here. Unified support for proprietary tooling > across subsystems and product lines for a given vendor. This way > migrating from in-tree networking to proprietary IPU/DPU networking > is easier, while migrating to another vendor would require full tooling > replacement. > > I have nothing against RDMA and CXL subsystems adding whatever APIs > they want. But I don't understand why you think it's okay to force > this on normal networking, which does not need it. > > nVidia is already refusing to add basic minoring features to their > upstream driver, and keeps asking its customers to migrate to libdoca. > So the concern that merging this will negatively impact standard > tooling is no longer theoretical. > > Anyway, rant over. Give us some technical details. The primary use-case that I find valuable is the ability to perform debug of different parts of a hardware pipeline when devices are already in the field. This could be the standard ethernet pipeline, RoCE, crypto, etc. We do have the ability to gather all the information we need via tools like ethtool and devlink, but there are cases where running a tool in real-time can help us know what is happening in a system on a per packet basis. We actually did something like this this week. When I look at fwctl, I don't see it as something that is valuable only today -- I see it as something that is valuable 2 years from now. When someone is still running v6.17 and we have discovered that a debug counter/infra that was added in v7.0, but they cannot use it without installing a new kernel or an OOB driver we don't have an option to easily help narrow down the problem. If a fairly simple tool can help perform RPC to FW to glean some of this hardware information we save days of back and forth debugging with special drivers to try and help narrow down the issue.
