On Sat, Jul 28, 2018 at 11:57:27PM +0300, Yuval Shaia wrote: > On Tue, Jul 24, 2018 at 03:43:33PM -0600, Jason Gunthorpe wrote: > > On Mon, Jul 16, 2018 at 11:03:53PM +0300, Yuval Shaia wrote: > > > RDMA MAD kernel module (ibcm) disallow more than one MAD-agent for a > > > given MAD class. > > > This does not go hand-by-hand with qemu pvrdma device's requirements > > > where each VM is MAD agent. > > > Fix it by adding implementation of RDMA MAD multiplexer service which on > > > one hand register as a sole MAD agent with the kernel module and on the > > > other hand gives service to more than one VM. > > > > > > Design Overview: > > > A server process is registered to UMAD framework (for this to work the > > > rdma_cm kernel module needs to be unloaded) and creates a unix socket to > > > listen to incoming request from clients. > > > A client process (such as QEMU) connects to this unix socket and > > > registers with its own GID. > > > > > > TX: > > > When client needs to send rdma_cm MAD message it construct it the same > > > way as without this multiplexer, i.e. creates a umad packet but this > > > time it writes its content to the socket instead of calling umad_send(). > > > The server, upon receiving such a message fetch local_comm_id from it so > > > a context for this session can be maintain and relay the message to UMAD > > > layer by calling umad_send(). > > > > > > RX: > > > The server creates a worker thread to process incoming rdma_cm MAD > > > messages. When an incoming message arrived (umad_recv()) the server, > > > depending on the message type (attr_id) looks for target client by > > > either searching in gid->fd table or in local_comm_id->fd table. With > > > the extracted fd the server relays to incoming message to the client. > > > > Eh? The entire reason the kernel forbis multiple registrations is > > because it has no way to de-multiplex them.. > > > > If you want to de-multiplex based on GRH.GID, or DMAC, we could > > probably arrange for the kernel to support that kind of model.. > > Yeah, this is exactly the missing functionality that we need. > > > > > Maybe it is even already done via the steering APIs? > > From user-space MAD agent? You'd have to user verbs, and steering and I don't know if it works with QP1 traffic, but it would be very clean.. Jason -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
