First patch adds custom getsockopt for TCP_ZEROCOPY_RECEIVE
to remove kmalloc and lock_sock overhead from the dat path.
Second patch removes kzalloc/kfree from getsockopt for the common cases.
Third patch switches cgroup_bpf_enabled to be per-attach to
to add only overhead for the cgroup attach types used on the system.
No visible user-side changes.
v9:
- include linux/tcp.h instead of netinet/tcp.h in sockopt_sk.c
- note that v9 depends on the commit 4be34f3d0731 ("bpf: Don't leak
memory in bpf getsockopt when optlen == 0") from bpf tree
v8:
- add bpi.h to tools/include/uapi in the same patch (Martin KaFai Lau)
- kmalloc instead of kzalloc when exporting buffer (Martin KaFai Lau)
- note that v8 depends on the commit 4be34f3d0731 ("bpf: Don't leak
memory in bpf getsockopt when optlen == 0") from bpf tree
v7:
- add comment about buffer contents for retval != 0 (Martin KaFai Lau)
- export tcp.h into tools/include/uapi (Martin KaFai Lau)
- note that v7 depends on the commit 4be34f3d0731 ("bpf: Don't leak
memory in bpf getsockopt when optlen == 0") from bpf tree
v6:
- avoid indirect cost for new bpf_bypass_getsockopt (Eric Dumazet)
v5:
- reorder patches to reduce the churn (Martin KaFai Lau)
v4:
- update performance numbers
- bypass_bpf_getsockopt (Martin KaFai Lau)
v3:
- remove extra newline, add comment about sizeof tcp_zerocopy_receive
(Martin KaFai Lau)
- add another patch to remove lock_sock overhead from
TCP_ZEROCOPY_RECEIVE; technically, this makes patch #1 obsolete,
but I'd still prefer to keep it to help with other socket
options
v2:
- perf numbers for getsockopt kmalloc reduction (Song Liu)
- (sk) in BPF_CGROUP_PRE_CONNECT_ENABLED (Song Liu)
- 128 -> 64 buffer size, BUILD_BUG_ON (Martin KaFai Lau)
Stanislav Fomichev (3):
bpf: remove extra lock_sock for TCP_ZEROCOPY_RECEIVE
bpf: try to avoid kzalloc in cgroup/{s,g}etsockopt
bpf: split cgroup_bpf_enabled per attach type
include/linux/bpf-cgroup.h | 63 ++--
include/linux/filter.h | 5 +
include/linux/indirect_call_wrapper.h | 6 +
include/net/sock.h | 2 +
include/net/tcp.h | 1 +
kernel/bpf/cgroup.c | 112 +++++-
net/ipv4/af_inet.c | 9 +-
net/ipv4/tcp.c | 14 +
net/ipv4/tcp_ipv4.c | 1 +
net/ipv4/udp.c | 7 +-
net/ipv6/af_inet6.c | 9 +-
net/ipv6/tcp_ipv6.c | 1 +
net/ipv6/udp.c | 7 +-
net/socket.c | 3 +
tools/include/uapi/linux/tcp.h | 357 ++++++++++++++++++
.../selftests/bpf/prog_tests/bpf_tcp_ca.c | 1 +
.../selftests/bpf/prog_tests/cls_redirect.c | 1 +
.../selftests/bpf/prog_tests/sockmap_basic.c | 1 +
.../selftests/bpf/prog_tests/sockopt_sk.c | 28 ++
.../testing/selftests/bpf/progs/sockopt_sk.c | 23 +-
tools/testing/selftests/bpf/test_progs.h | 1 -
21 files changed, 597 insertions(+), 55 deletions(-)
create mode 100644 tools/include/uapi/linux/tcp.h
--
2.30.0.284.gd98b1dd5eaa7-goog
