> On 25 Nov 2020, at 17:43, Christopher Lameter <cl@xxxxxxxxx> wrote: > > On Wed, 25 Nov 2020, Honggang LI wrote: > >>> How do I figure out why ibacm is not talking to the subnet manager? >> >> No, you can't talking to subnet manager, if you resolve IPoIB IP address >> or hostname to PathRecord. The query MAD packets will be send to one >> multicast group all ibacm service attached. > > Huh? When does it talk to a subnet manager (or the SA)? When resolving the route AND the option "route_prot" is set to "sa". If set to "acm", what Hong describes above applies. > If its get an IP address of an IB node that does not have ibacm then it > fails with a timeout ..... ? And leaves hanging kernel threads around by > design? Nop, the kernel falls back and uses the neighbour cache instead. > So it only populates the cache from its local node information? No, if you use ibacm for address resolution the only protocol it has is "acm", which means the information comes from a peer ibacm. If you talk about the cache for routes, it comes either from the SA or a peer ibacm, depending on the "route_prot" setting. >> To resolve IPoIB address to PathRecord, you must: >> 1) The IPoIB interface must UP and RUNNING on the client and target >> side. >> 2) The ibacm service must RUNNING on the client and target. > > That is working if you want to resolve only the IP addresses of the IB > interfaces on the client and target. None else. That is why it is called IBacm, right? > Here is the description of ibacms function from the sources: > > "Conceptually, the ibacm service implements an ARP like protocol and > either uses IB multicast records to construct path record data or queries > the SA directly, depending on the selected route protocol. By default, the > ibacm services uses and caches SA path record queries." > > SA queries dont work. So its broken and cannot talk to the SM. Why do you say that? It works all the time for me which uses "sa" as "route_prot". Thxs, Håkon
