Selvin Xavier <selvin.xavier@xxxxxxxxxxxx> wrote on 13/06/2024 at 10:49:00+0200: > On Wed, Jun 12, 2024 at 11:59 PM Pierre-Elliott Bécue <peb@xxxxxxxxxx> wrote: > > De : Selvin Xavier <selvin.xavier@xxxxxxxxxxxx> > À : Pierre-Elliott Bécue <peb@xxxxxxxxxx> > Cc : linux-rdma@xxxxxxxxxxxxxxx > Date : 12 juin 2024 19:18:34 > Objet : Re: What's the current status with bnxt_re-abi.h > > > Hi, > > bnxt_re-abi.h in linux kernel and rdma-core uses abi version 1. We > > dont bump up the version in upstream and backward compatibility is > > maintained using the comp_mask field of the interface structures. > > > > If you are using the latest drivers maintained in the Broadcom site > > (which uses ABI version 6), you need to use the libbnxt_re hosted in > > the Broacom site itself. We maintain compatibility between the Out of > > tree driver and Out of tree library. > > > > Thanks, > > Selvin > > > > On Wed, Jun 12, 2024 at 10:21 PM Pierre-Elliott Bécue <peb@xxxxxxxxxx> wrote: > >> > >> Pierre-Elliott Bécue <peb@xxxxxxxxxx> wrote on 12/06/2024 at 18:47:36+0200: > >> > >>> Hello, > >>> > >>> In bnxt_re-abi.h, the abi version mentioned is 1. It's used as it's in > >>> all libibverbs to determine the min AND max supported ABI. > >>> > >>> bnxt_re isn't currently mainlined in the kernel, > >> > >> Sorry, a word is missing: "Recent bnxt_re isn't currently mainlined" > >> > >>> and those eager to use > >>> the driver need to rely on the one provided by broadcom on their > >>> website. > >>> > >>> The thing is, they bumped their ABI version multiple times (current is > >>> 6). In the current context, one can't use the manually compiled bnxt_re > >>> driver with libibverbs as any call will error due to the bnxt_re abi > >>> version being outside of min/max supported abi version. > >>> > >>> What's the current situation regarding bnxt_re, should we consider > >>> libibverb support of bnxt_re as deprecated? > >>> > >>> Of course I could have missed something, sorry for that if that's the > >>> case. > >>> > >>> Bests, > Hey Selvin, > > Thanks a lot for clarifying this. > > I built libbnxt_re, but the thing is, I can't use the usual infiniband tooling (ib_write_bw) et al without using libibverbs which is still the > linux-rdma one. > > Do you have an alternative to suggest that I should consider? > > It should work fine with the libibverbs installed by > linux-rdma-core. Due to the ABI version being 1 in rdma-core, no. libibverbs verifies that the current driver ABI version (6 for bnxt_re from broadcom's website) is between ops->match_min_abi_version and ops->match_max_abi_version, both values for bnxt_re are set to the ABI version (1). https://github.com/linux-rdma/rdma-core/blob/master/libibverbs/init.c#L363-L372 So when trying to use tooling from rdma-core (like ib_write_bw), the match_device function returns false and the card can't be used. > The Broadcom support team can help you with the > necessary instructions. I will forward your mail to them and they will > help. I already interacted with the support team, and I'm not confident they can help. But thanks. :) > Also, would a RDMA+SRIOV-capable broadcom card with the latest firmware (229 something) work fine with the drivers shipped in > rdma-core. > > If so maybe I should use what's provided by the kernel and rdma-core. > > The drivers shipped with rdma-core and kernel should work for most of > your use cases. I suggest you try this. Ack, I'll try and compare perfs. Thanks! -- PEB
Attachment:
signature.asc
Description: PGP signature
