On Fri, Apr 19, 2024 at 3:19 PM Xin Long <lucien.xin@xxxxxxxxx> wrote: > > On Fri, Apr 19, 2024 at 2:51 PM Stefan Metzmacher <metze@xxxxxxxxx> wrote: > > > > Hi Xin Long, > > > > >> But I think its unavoidable for the ALPN and SNI fields on > > >> the server side. As every service tries to use udp port 443 > > >> and somehow that needs to be shared if multiple services want to > > >> use it. > > >> > > >> I guess on the acceptor side we would need to somehow detach low level > > >> udp struct sock from the logical listen struct sock. > > >> > > >> And quic_do_listen_rcv() would need to find the correct logical listening > > >> socket and call quic_request_sock_enqueue() on the logical socket > > >> not the lowlevel udo socket. The same for all stuff happening after > > >> quic_request_sock_enqueue() at the end of quic_do_listen_rcv. > > >> > > > The implementation allows one low level UDP sock to serve for multiple > > > QUIC socks. > > > > > > Currently, if your 3 quic applications listen to the same address:port > > > with SO_REUSEPORT socket option set, the incoming connection will choose > > > one of your applications randomly with hash(client_addr+port) vi > > > reuseport_select_sock() in quic_sock_lookup(). > > > > > > It should be easy to do a further match with ALPN between these 3 quic > > > socks that listens to the same address:port to get the right quic sock, > > > instead of that randomly choosing. > > > > Ah, that sounds good. > > > > > The problem is to parse the TLS Client_Hello message to get the ALPN in > > > quic_sock_lookup(), which is not a proper thing to do in kernel, and > > > might be rejected by networking maintainers, I need to check with them. > > > > Is the reassembling of CRYPTO frames done in the kernel or > > userspace? Can you point me to the place in the code? > In quic_inq_handshake_tail() in kernel, for Client Initial packet > is processed when calling accept(), this is the path: > > quic_accept()-> quic_accept_sock_init() -> quic_packet_process() -> > quic_packet_handshake_process() -> quic_frame_process() -> > quic_frame_crypto_process() -> quic_inq_handshake_tail(). > > Note that it's with the accept sock, not the listen sock. > > > > > If it's really impossible to do in C code maybe > > registering a bpf function in order to allow a listener > > to check the intial quic packet and decide if it wants to serve > > that connection would be possible as last resort? > That's a smart idea! man. > I think the bpf hook in reuseport_select_sock() is meant to do such > selection. > > For the Client initial packet (the only packet you need to handle), > I double you will need to do the reassembling, as Client Hello TLS message > is always less than 400 byte in my env. > > But I think you need to do the decryption for the Client initial packet > before decoding it then parsing the TLS message from its crypto frame. I created this patch: https://github.com/lxin/quic/commit/aee0b7c77df3f39941f98bb901c73fdc560befb8 to do this decryption in quic_sock_look() before calling reuseport_select_sock(), so that it provides the bpf selector with a plain-text QUIC initial packet: https://datatracker.ietf.org/doc/html/rfc9000#section-17.2.2 If it's complex for you to do the decryption for the initial packet in the bpf selector, I will apply this patch. Please let me know. Thanks. > > BTW, for the TLS message parse, I have some prototype code for > TLS Handshake: > https://github.com/lxin/tls_hs/blob/master/crypto/tls_hs.c#L2084 > > The path to get ALPN: > tls_msg_handle() -> tls_msg_ch_handle() -> tls_ext_handle() > > Hope it may be helpful to you. > > > > > > Will you be able to work around this by using Unix Domain Sockets pass > > > the sockfd to another process? > > > > Not really. As that would strict coordination between a lot of > > independent projects. > > > > > (Note that we're assuming all your 3 applications are using in-kernel QUIC) > > > > Sure, but I guess for servers using port 443 that the only long term option. > > and I don't think it will be less performant than a userspace implementation. > Cool.
