Hi Daniel, On Tue, Jul 28, 2020 at 10:47 PM CEST, Daniel Borkmann wrote: [...] > Jakub, I'm actually seeing a slightly different one on my test machine with sk_lookup: > > # ./test_progs -t sk_lookup > #14 cgroup_skb_sk_lookup:OK > #73/1 query lookup prog:OK > #73/2 TCP IPv4 redir port:OK > #73/3 TCP IPv4 redir addr:OK > #73/4 TCP IPv4 redir with reuseport:OK > #73/5 TCP IPv4 redir skip reuseport:OK > #73/6 TCP IPv6 redir port:OK > #73/7 TCP IPv6 redir addr:OK > #73/8 TCP IPv4->IPv6 redir port:OK > #73/9 TCP IPv6 redir with reuseport:OK > #73/10 TCP IPv6 redir skip reuseport:OK > #73/11 UDP IPv4 redir port:OK > #73/12 UDP IPv4 redir addr:OK > #73/13 UDP IPv4 redir with reuseport:OK > attach_lookup_prog:PASS:open 0 nsec > attach_lookup_prog:PASS:bpf_program__attach_netns 0 nsec > make_socket:PASS:make_address 0 nsec > make_socket:PASS:socket 0 nsec > make_socket:PASS:setsockopt(SO_SNDTIMEO) 0 nsec > make_socket:PASS:setsockopt(SO_RCVTIMEO) 0 nsec > make_server:PASS:setsockopt(IP_RECVORIGDSTADDR) 0 nsec > make_server:PASS:setsockopt(SO_REUSEPORT) 0 nsec > make_server:PASS:bind 0 nsec > make_server:PASS:attach_reuseport 0 nsec > update_lookup_map:PASS:bpf_map__fd 0 nsec > update_lookup_map:PASS:bpf_map_update_elem 0 nsec > make_socket:PASS:make_address 0 nsec > make_socket:PASS:socket 0 nsec > make_socket:PASS:setsockopt(SO_SNDTIMEO) 0 nsec > make_socket:PASS:setsockopt(SO_RCVTIMEO) 0 nsec > make_server:PASS:setsockopt(IP_RECVORIGDSTADDR) 0 nsec > make_server:PASS:setsockopt(SO_REUSEPORT) 0 nsec > make_server:PASS:bind 0 nsec > make_server:PASS:attach_reuseport 0 nsec > update_lookup_map:PASS:bpf_map__fd 0 nsec > update_lookup_map:PASS:bpf_map_update_elem 0 nsec > make_socket:PASS:make_address 0 nsec > make_socket:PASS:socket 0 nsec > make_socket:PASS:setsockopt(SO_SNDTIMEO) 0 nsec > make_socket:PASS:setsockopt(SO_RCVTIMEO) 0 nsec > make_server:PASS:setsockopt(IP_RECVORIGDSTADDR) 0 nsec > make_server:PASS:setsockopt(SO_REUSEPORT) 0 nsec > make_server:PASS:bind 0 nsec > make_server:PASS:attach_reuseport 0 nsec > run_lookup_prog:PASS:getsockname 0 nsec > run_lookup_prog:PASS:connect 0 nsec > make_socket:PASS:make_address 0 nsec > make_socket:PASS:socket 0 nsec > make_socket:PASS:setsockopt(SO_SNDTIMEO) 0 nsec > make_socket:PASS:setsockopt(SO_RCVTIMEO) 0 nsec > make_client:PASS:make_client 0 nsec > send_byte:PASS:send_byte 0 nsec > udp_recv_send:FAIL:recvmsg failed > (/root/bpf-next/tools/testing/selftests/bpf/prog_tests/sk_lookup.c:339: errno: Resource temporarily unavailable) failed to receive > #73/14 UDP IPv4 redir and reuseport with conns:FAIL > #73/15 UDP IPv4 redir skip reuseport:OK > #73/16 UDP IPv6 redir port:OK > #73/17 UDP IPv6 redir addr:OK > #73/18 UDP IPv4->IPv6 redir port:OK > #73/19 UDP IPv6 redir and reuseport:OK > attach_lookup_prog:PASS:open 0 nsec > attach_lookup_prog:PASS:bpf_program__attach_netns 0 nsec > make_socket:PASS:make_address 0 nsec > make_socket:PASS:socket 0 nsec > make_socket:PASS:setsockopt(SO_SNDTIMEO) 0 nsec > make_socket:PASS:setsockopt(SO_RCVTIMEO) 0 nsec > make_server:PASS:setsockopt(IP_RECVORIGDSTADDR) 0 nsec > make_server:PASS:setsockopt(IPV6_RECVORIGDSTADDR) 0 nsec > make_server:PASS:setsockopt(SO_REUSEPORT) 0 nsec > make_server:PASS:bind 0 nsec > make_server:PASS:attach_reuseport 0 nsec > update_lookup_map:PASS:bpf_map__fd 0 nsec > update_lookup_map:PASS:bpf_map_update_elem 0 nsec > make_socket:PASS:make_address 0 nsec > make_socket:PASS:socket 0 nsec > make_socket:PASS:setsockopt(SO_SNDTIMEO) 0 nsec > make_socket:PASS:setsockopt(SO_RCVTIMEO) 0 nsec > make_server:PASS:setsockopt(IP_RECVORIGDSTADDR) 0 nsec > make_server:PASS:setsockopt(IPV6_RECVORIGDSTADDR) 0 nsec > make_server:PASS:setsockopt(SO_REUSEPORT) 0 nsec > make_server:PASS:bind 0 nsec > make_server:PASS:attach_reuseport 0 nsec > update_lookup_map:PASS:bpf_map__fd 0 nsec > update_lookup_map:PASS:bpf_map_update_elem 0 nsec > make_socket:PASS:make_address 0 nsec > make_socket:PASS:socket 0 nsec > make_socket:PASS:setsockopt(SO_SNDTIMEO) 0 nsec > make_socket:PASS:setsockopt(SO_RCVTIMEO) 0 nsec > make_server:PASS:setsockopt(IP_RECVORIGDSTADDR) 0 nsec > make_server:PASS:setsockopt(IPV6_RECVORIGDSTADDR) 0 nsec > make_server:PASS:setsockopt(SO_REUSEPORT) 0 nsec > make_server:PASS:bind 0 nsec > make_server:PASS:attach_reuseport 0 nsec > run_lookup_prog:PASS:getsockname 0 nsec > run_lookup_prog:PASS:connect 0 nsec > make_socket:PASS:make_address 0 nsec > make_socket:PASS:socket 0 nsec > make_socket:PASS:setsockopt(SO_SNDTIMEO) 0 nsec > make_socket:PASS:setsockopt(SO_RCVTIMEO) 0 nsec > make_client:PASS:make_client 0 nsec > send_byte:PASS:send_byte 0 nsec > udp_recv_send:FAIL:recvmsg failed > (/root/bpf-next/tools/testing/selftests/bpf/prog_tests/sk_lookup.c:339: errno: Resource temporarily unavailable) failed to receive > #73/20 UDP IPv6 redir and reuseport with conns:FAIL > #73/21 UDP IPv6 redir skip reuseport:OK > #73/22 TCP IPv4 drop on lookup:OK > #73/23 TCP IPv6 drop on lookup:OK > #73/24 UDP IPv4 drop on lookup:OK > #73/25 UDP IPv6 drop on lookup:OK > #73/26 TCP IPv4 drop on reuseport:OK > #73/27 TCP IPv6 drop on reuseport:OK > #73/28 UDP IPv4 drop on reuseport:OK > #73/29 TCP IPv6 drop on reuseport:OK > #73/30 sk_assign returns EEXIST:OK > #73/31 sk_assign honors F_REPLACE:OK > #73/32 sk_assign accepts NULL socket:OK > #73/33 access ctx->sk:OK > #73/34 narrow access to ctx v4:OK > #73/35 narrow access to ctx v6:OK > #73/36 sk_assign rejects TCP established:OK > #73/37 sk_assign rejects UDP connected:OK > #73/38 multi prog - pass, pass:OK > #73/39 multi prog - drop, drop:OK > #73/40 multi prog - pass, drop:OK > #73/41 multi prog - drop, pass:OK > #73/42 multi prog - pass, redir:OK > #73/43 multi prog - redir, pass:OK > #73/44 multi prog - drop, redir:OK > #73/45 multi prog - redir, drop:OK > #73/46 multi prog - redir, redir:OK > #73 sk_lookup:FAIL > Summary: 1/44 PASSED, 0 SKIPPED, 3 FAILED This patch addresses the failures: https://lore.kernel.org/bpf/20200726120228.1414348-1-jakub@xxxxxxxxxxxxxx/ It spawned a discussion on what to do about reuseport bpf returning connected udp sockets, and the conclusion was that it would be best to change reuseport logic itself if no one is relying on said behavior. IOW, I belive the fix does the right thing and can be applied as is. We get the same reuseport behavior everywhere, that is with regular socket lookup and BPF sk lookup, even if that behavior needs to be changed. [...]
