在 2024/7/30 1:10, Andrew Sheinberg 写道:
Hello all,
I’m not sure if this the right place to ask, but I will give it a try.
I have a system with many initialized UD queue pairs (info for address handle creations and qp numbers exchanged out-of-band). I am only using libibverbs for establishment (purposefully not using librdmacm, to allow for more flexible environment configuration) — everything is working smoothly for unicast. Now I would like to create a multicast group and attach some of these queue pairs (ibv_mcast_attach); however I am struggling to find any details on how to create such a group (and obtain a proper MGID and MLID).
I found a few examples online but am left with questions:
- There is code within perftest's "multicast_resources.c", but this seems a bit hacky and oddly verbose
- There is code within Nvidia Docs’ "Programming Examples using IBV” showcasing joining an already created multicast group at a given IP address using rdma_cm, but It is unclear how to create the group in the first place
Questions (please correct me if these do not make sense):
1. What is the role of the OpenSM — is there a C API?
- Are there any examples using opensm programmatically and not with CLI?
- Does the API differ on InfiniBand vs. RoCEv2 fabric?
I have made tests with Infiniband vs. RoCEv2. From my perspective, I
think, there are some differences between the 2 fabrics.
To the basic functionalities, the difference between 2 fabrics is small.
But to the extended functionalities, the difference is big.
The difference is based on the different features.
I just made some simple tests. Perhaps some engineers who made a lot of
tests can give more suggestions about this.
Zhu Yanjun
2. Is there any high-level documentation to describe the role of libibumad? (Looking at the man pages on a per-function basis is a bit too fine-grained for my understanding as of now).
- I also see libibmad — what is the responsibility breakdown between these two?
- How do they relate to OpenSM?
Any guidance is greatly appreciated.
Thanks,
Andrew Sheinberg
