> On Apr 27, 2022, at 7:53 PM, Jakub Kicinski <kuba@xxxxxxxxxx> wrote: > > On Wed, 27 Apr 2022 14:42:53 +0000 Chuck Lever III wrote: >>> On Apr 26, 2022, at 7:47 PM, Jakub Kicinski <kuba@xxxxxxxxxx> wrote: >>>> RPC-with-TLS requires one RPC as a "starttls" token. That could be >>>> done in user space as part of the handshake, but it is currently >>>> done in the kernel to enable the user agent to be shared with other >>>> kernel consumers of TLS. Keep in mind that we already have two >>>> real consumers: NVMe and RPC-with-TLS; and possibly QUIC. >>>> >>>> You asserted earlier that creating sockets in user space "scales >>>> better" but did not provide any data. Can we see some? How well >>>> does it need to scale for storage protocols that use long-lived >>>> connections? >>> >>> I meant scale with the number of possible crypto protocols, >>> I mentioned three there. >> >> I'm looking at previous emails. The "three crypto protocols" >> don't stand out to me. Which ones? > > TLS, QUIC and PSP maybe that was in a different email that what you > quoted, sorry: > https://lore.kernel.org/all/20220426080247.19bbb64e@xxxxxxxxxx/ > > PSP: > https://raw.githubusercontent.com/google/psp/main/doc/PSP_Arch_Spec.pdf During the design process, we discussed both TLS and QUIC handshake requirements, which are nearly the same. QUIC will want a TLSv1.3 handshake on a UDP socket, effectively. We can support DTLS in a similar fashion. We hope that the proposed design can be used for all of those, and barring anything unforeseen in the description of PSP you provided, PSP can be supported as well. The handshake agent is really only a shim around a TLS library. There isn't much to it. > Is it possible to instead create a fd-passing-like structured message > which could carry the fd and all the relevant context (what goes > via the getsockopt() now)? > > The user space agent can open such upcall socket, then bind to > whatever entity it wants to talk to on the kernel side and read > the notifications via recv()? We considered this kind of design. A reasonable place to start there would be to fabricate new NETLINK messages to do this. I don't see much benefit over what is done now, it's just a different isomer of syntactic sugar, but it could be considered. The issue is how the connected socket is materialized in user space. accept(2) is the historical way to instantiate an already connected socket in a process's file table, and seems like a natural fit. When the handshake agent is done with the handshake, it closes the socket. This invokes the tlsh_release() function which can check whether the IV implantation was successful. So instead of an AF_TLSH listener we could use a named pipe or a netlink socket and a blocking recv(), as long as there is a reasonable solution to how a connected socket fd is attached to the handshake agent process. I'm flexible about the mechanism for passing handshake parameters. Attaching them to the connected socket seems convenient, but perhaps not aesthetic. -- Chuck Lever
