On 8/20/21 4:18 AM, Stefan Metzmacher wrote: > Hi Dennis, > > just as a wild idea, would be an option to use the SMB-Direct [1] protocol defined here? > > It basically provides a stream/packet like transport based on IB_WR_SEND[_WITH_INV] > and in addition it allows direct memory transfers with IB_WR_RDMA_READ or IB_WR_RDMA_WRITE. > > It's called SMB-Direct as it's currently used as a transport for the SMB3 protocol, > but it could also be used as transport for other things. > > Over the last years I've been working on a PF_SMBDIRECT socket driver [2] > as a hobby project in order to support it for Samba. It's not yet production ready > and has known memory leaks, but the basics already work. The api [3] is based on > sendmsg/recvmsg with using MSG_OOB with msg->msg_control for direct memory transfers. > I'll actually use it with IORING_OP_SENDMSG and IORING_OP_RECVMSG, which allow msg->msg_control > starting 5.12 kernels. > > metze > > [1] https://winprotocoldoc.blob.core.windows.net/productionwindowsarchives/MS-SMBD/%5bMS-SMBD%5d.pdf > [2] https://git.samba.org/?p=metze/linux/smbdirect.git;a=summary > [3] https://git.samba.org/?p=metze/linux/smbdirect.git;a=blob;f=smbdirect.h;hb=refs/heads/smbdirect-work-in-progress > The problem with SMB-Direct is probably going to be the same as with things like RDS in that there is extra overhead beyond what we are trying to achieve. I can honestly say it's not something I've considered so I will definitely take some time to read through the links you provided. Thanks -Denny
