On Tue, 2015-05-05 at 19:27 +0000, Liran Liss wrote: > > From: linux-rdma-owner@xxxxxxxxxxxxxxx [mailto:linux-rdma- > > > owner@xxxxxxxxxxxxxxx] On Behalf Of Hefty, Sean > > > > > > In accordance with what we've been talking about, drop IBOE for ROCE. > > > > > > Drop the UDP off of USNIC, then define a bit for CAP_PROT_UDP_ENCAP. > > > USNIC will be just USNIC, USNIC_UDP will be USNIC | UDP_ENCAP, ROCE v1 > > > will be ROCE, and ROCEv2 will be ROCE | UDP_ENCAP. > > > > USNIC_UDP is just UDP. I don't understand why we would want 'USNIC | > > UDP_ENCAP', or what UDP_ENCAP is intended to convey. Nothing is being > > encapsulated. > > > > I agree that the UDP_ENCAP notion is confusing. > We should stick to ROCE_V1 and ROCE_V2. > > > RoCEv2 is IB transport over UDP. > > > > I'm not sure what the protocol field is intended to imply. If we want to > > expose the link, network, transport, and RDMA protocols in use, shouldn't > > these be separate fields or bits? And even then, I'm not sure what use this > > has for the ULPs. iWarp does not require Ethernet or TCP. RoCEv2 would > > work fine over any link. And the core layer should not assume that a device is > > limited to supporting only one protocol, especially at the network and > > transport levels. I vote for deprecating the protocol goofiness. > > > > - Sean > > The protocol notion might not have any value for ULPs, but it is useful for core > management code. Also, when we extend these notions to user-space, admins will > actually want to know what wire protocols a certain device can support, even > just for the sake of interoperability. So I've been thinking a bit more about the overall architecture of the underlying patch set from Michael. The structure of that patch set is to a certain extent dictating this follow on patch set and so any problems in it end up being transmitted here as well. Sean's comments about transports I think were spot on, and they got me doing a little thought exercise in my head. Theoretically, if we were to take the SoftiWARP and SoftRoCE drivers, we could merge them down to one driver that supported both iWARP and RoCE (and we could add USNIC too I suspect). Because all of these things work on an Ethernet link layer, there is no reason they couldn't all be presented on the same device. So if you had a single verbs device that could support all of these things, where would you need to capture and store the relevant information? This is what I have so far: On a per device basis: not much to be honest, node_type IB_CA/RNIC for back compatibility and that's pretty much it On a per port basis: link layer (Ethernet, IB, or OPA). Notice that I didn't include the transport here. Two of the above link layers strictly imply their transport. The third link layer could have multiple transports. Which brings me to my next level... On a per queue pair basis: if (link_layer == Ethernet) iWARP/RoCE/USNIC if (qp_type == RoCE && qp != UD) RoCEv1/RoCEv2 On a per ah basis: if (qp_type == RoCE && qp = UD) RoCEv1/RoCEv2 Michael's patch set was intended to give us a framework within which we can start cleaning things up. Once we get to cleaning things up, I think the above is where we should target storing the relevant information. Thoughts? -- Doug Ledford <dledford@xxxxxxxxxx> GPG KeyID: 0E572FDD
Attachment:
signature.asc
Description: This is a digitally signed message part
