On Mon, Apr 29, 2024 at 11:20 AM Stefan Metzmacher <metze@xxxxxxxxx> wrote: > > Hi Xin Long, > > >> > > Just confirmed from other ebpf experts, there are no in-kernel interfaces > > for loading and interacting with BPF maps/programs(other than from BPF itself). > > > > It seems that we have to do this match in QUIC stack. In the latest QUIC > > code, I added quic_packet_get_alpn(), a 59-line function, to parse ALPNs > > and then it will search for the listen sock with these ALPNs in > > quic_sock_lookup(). > > > > I introduced 'alpn_match' module param, and it can be enabled when loading > > the module QUIC by: > > > > # modprobe quic alpn_match=1 > > > > You can test it by tests/sample_test in the latest code: > > > > Start 3 servers: > > > > # ./sample_test server 0.0.0.0 1234 \ > > ./keys/server-key.pem ./keys/server-cert.pem smbd > > # ./sample_test server 0.0.0.0 1234 \ > > ./keys/server-key.pem ./keys/server-cert.pem h3 > > # ./sample_test server 0.0.0.0 1234 \ > > ./keys/server-key.pem ./keys/server-cert.pem ksmbd > > > > Try to connect on clients with: > > > > # ./sample_test client 127.0.0.1 1234 ksmbd > > # ./sample_test client 127.0.0.1 1234 smbd > > # ./sample_test client 127.0.0.1 1234 h3 > > > > to see if the corresponding server responds. > > > > There might be some concerns but it's also a useful feature that can not > > be implemented in userland QUICs. The commit is here: > > > > https://github.com/lxin/quic/commit/de82f8135f4e9196b503b4ab5b359d88f2b2097f > > > > Please check if this is enough for SMB applications. > > It look great thanks! > > > Note as a listen socket is now identified by [address + port + ALPN] when > > alpn_match=1, this feature does NOT require SO_REUSEPORT socket option to > > be set, unless one wants multiple sockets to listen to > > the same [address + port + ALPN]. > > I'd argue that this should be the default and be required before listen() > or maybe before bind(), so that it can return EADDRINUSE. As EADDRINUSE should only > happen for servers it might be useful to have a QUIC_SOCKOPT_LISTEN_ALPN instead of > QUIC_SOCKOPT_ALPN. As QUIC_SOCKOPT_ALPN on a client socket should not generate let > bind() care about the alpn value at all. The latest patches have made it always do alpn_match in kernel, and also support multiple ALPNs(split by ',' when setting it via sockopt) on both server and client side. Feel free to check. Note that: 1. As you expected, setsockopt(QUIC_SOCKOPT_ALPN) must be called before listen(), and it will return EADDRINUSE if there's a socket already listening to the same IP + PORT + ALPN. 2. ALPN bind/match is a *listening* sockets thing, so it checks ALPN only when adding listening sockets in quic_hash(), and it does ALPN only when looking up listening sockets in quic_sock_lookup(). By setting ALPNs in client sockets it will ONLY pack these ALPNs into the Client Initial Packet when starting connecting, no bind/match for these regular sockets, as these sockets can be found by 4-tuple or a source_connection_id. bind() doesn't need to care about ALPN for client/regular socket either. So it's fine to use QUIC_SOCKOPT_ALPN sockopt for both listen and regular/client sockets, as in kernel it acts differently on ALPNs for listening and regular sockets. (sorry for confusing, I could have moved created another hashtable for listening sockets) In other word, a listen socket is identified by local_ip + local_port + ALPN(s) while a regular socket (represents a quic connection) is identified by: local_ip + local_port + remote_ip + remote_port or any of those source_connection_ids. 3. SO_REUSEPORT is still applied to do some load balance between multiple processes listening to the same IP + PORT + ALPN, like: on server: process A: skA = listen(127.0.0.1:1234:smbd) process B: skB = listen(127.0.0.1:1234:smbd) process C: skC = listen(127.0.0.1:1234:smbd) on client: connect(127.0.0.1:1234:smbd) connect(127.0.0.1:1234:smbd) ... on server it will select the sk among (skA, skB and skC) based on the source address + port in the request from client. 4. Not sure if multiple ALPNs support are useful to you, here is some example about how it works: - Without SO_REUSEPORT set: On server: process A: skA = listen(127.0.0.1:1234:smbd,h3,ksmbd) process B: skB = listen(127.0.0.1:1234:smbd,h3,ksmbd) listen() in process B fails and returns EADDRINUSE. - with SO_REUSEPORT set: On server: process A: skA = listen(127.0.0.1:1234:smbd,h3,ksmbd) process B: skB = listen(127.0.0.1:1234:smbd,h3,ksmbd) listen() in process B works. - with or without SO_REUSEPORT set: On server: process A: skA = listen(127.0.0.1:1234:h3,ksmbd) process B: skB = listen(127.0.0.1:1234:h3,smbd). (there's overlap on ALPN list but not exact the same ALPNs) listen() in process B fails and returns EADDRINUSE. - the match priority for multiple ALPNs is based on the order on the client ALPN list: On server: process A: skA = listen(127.0.0.1:1234:smbd) process B: skB = listen(127.0.0.1:1234:h3) process C: skC = listen(127.0.0.1:1234:ksmbd) On client: process X: skX = connect(27.0.0.1:1234:h3,ksmbd,smbd) skB will be the one selected to accept the connection, as h3 is the 1st ALPN on the client ALPN list 'h3,ksmbd,smbd'. > > For listens on tcp you also need to specify an explicit port (at least in order > to be useful). > > And it would mean that all application would use it and not block other applications > from using an explicit alpn. > > Also an module parameter for this means the administrator would have to take care > of it, which means it might be unuseable if loaded with it. Agree, already dropped this param. > > I hope to find some time in the next weeks to play with this. > Should be relatively trivial create a prototype for samba's smbd. Sounds Cool! Thanks.
