> On Mar 16, 2023, at 1:28 PM, David Howells <dhowells@xxxxxxxxxx> wrote: > > Chuck Lever III <chuck.lever@xxxxxxxxxx> wrote: > >> That means I haven't seen the cover letter and do not have any >> context for this proposed change. > > https://lore.kernel.org/linux-fsdevel/20230316152618.711970-1-dhowells@xxxxxxxxxx/ > >> We've tried combining the sendpages calls in here before. It >> results in a significant and measurable performance regression. >> See: >> >> da1661b93bf4 ("SUNRPC: Teach server to use xprt_sock_sendmsg for socket sends") > > The commit replaced the use of sendpage with sendmsg, but that took away the > zerocopy aspect of sendpage. The idea behind MSG_SPLICE_PAGES is that it > allows you to do keep that. I'll have to try reapplying this commit and > adding the MSG_SPLICE_PAGES flag. Note that, as Trond point out, NFSD can handle an NFS READ request with either a splice actor or by copying through a vector, depending on what the underlying filesystem can support and whether we are using a security flavor that requires stable pages. Grep for RQ_SPLICE_OK. Eventually we want to make use of iomaps to ensure that reading areas of a file that are not allocated on disk does not trigger an extent allocation. Anna is working on that, but I have no idea what it will look like. We can talk more at LSF, if you'll both be around. Also... I find I have to put back the use of MSG_MORE and friends in here, otherwise kTLS will split each of these kernel_sendsomething() calls into its own TLS record. This code is likely going to look different after support for RPC-with-TLS goes in. >> Therefore, this kind of change needs to be accompanied by both >> benchmark results and some field testing to convince me it won't >> cause harm. > > Yep. > >> And, we have to make certain that this doesn't break operation >> with kTLS sockets... do they support MSG_SPLICE_PAGES ? > > I haven't yet tackled AF_TLS, AF_KCM or AF_SMC as they seem significantly more > complex than TCP and UDP. I thought I'd get some feedback on what I have > before I tried my hand at those. OK, I didn't mean AF_TLS, I meant the stuff under net/tls, which is AF_INET[6] and TCP, but with a ULP in place. It's got its own sendpage and sendmsg methods that choke when an unrecognized MSG_ flag is present. But OK, you're just asking for feedback, so I'll put my red pencil down. -- Chuck Lever
