Hi Pablo,
On Thu, Aug 27, 2020 at 07:28:42PM +0200, Pablo Neira Ayuso wrote:
> On x86_64, each notification results in one skbuff allocation which
> consumes at least 768 bytes due to the skbuff overhead.
>
> This patch coalesces several notifications into one single skbuff, so
> each notification consumes at least ~211 bytes, that ~3.5 times less
> memory consumption. As a result, this is reducing the chances to exhaust
> the netlink socket receive buffer.
>
> Rule of thumb is that each notification batch only contains netlink
> messages whose report flag is the same, nfnetlink_send() requires this
> to do appropriately delivery to userspace, either via unicast (echo
> mode) or multicast (monitor mode).
>
> The skbuff control buffer is used to annotate the report flag for later
> handling at the new coalescing routine.
>
> The batch skbuff notification size is NLMSG_GOODSIZE, using a larger
> skbuff would allow for more socket receiver buffer savings (to amortize
> the cost of the skbuff even more), however, going over that size might
> break userspace applications, so let's be conservative and stick to
> NLMSG_GOODSIZE.
With this patch in place on top of your other one ("netfilter:
nfnetlink: nfnetlink_unicast() reports EAGAIN instead of ENOBUFS"),
firewalld (as well as nft with same input) now report:
| netlink: Error: Could not process rule: No space left on device
The JSON snippet causing the problem is indeed quite big though:
| % grep "\"\(add\|insert\)\"" fail.pp.json | wc -l
| 462
Eric told me he plans to split initial ruleset creation into several
chunks though, so this should at least not be a blocker for firewalld.
something seems to be fishy, though. Here's a reproducer:
| #!/bin/bash
|
| numrules="$1"
|
| nft flush ruleset
| (
| echo "add table t"
| echo "add chain t c"
| for ((i = 0; i < $numrules; i++)); do
| echo "add rule t c ip saddr 10.0.0.1 ip daddr 10.0.0.2 tcp dport 27374 mark 0x23 counter accept"
| done
| ) | nft -ef -
It starts failing at 13, which is not much. Interestingly, it fails outside of
the container, too. And it even echoes part of the commands:
| add table ip t
| add chain ip t c
| add rule ip t c ip saddr 10.0.0.1 ip daddr 10.0.0.2 tcp dport 27374 meta mark 0x00000023 counter packets 0 bytes 0 accept
| add rule ip t c ip saddr 10.0.0.1 ip daddr 10.0.0.2 tcp dport 27374 meta mark 0x00000023 counter packets 0 bytes 0 accept
| add rule ip t c ip saddr 10.0.0.1 ip daddr 10.0.0.2 tcp dport 27374 meta mark 0x00000023 counter packets 0 bytes 0 accept
| add rule ip t c ip saddr 10.0.0.1 ip daddr 10.0.0.2 tcp dport 27374 meta mark 0x00000023 counter packets 0 bytes 0 accept
| add rule ip t c ip saddr 10.0.0.1 ip daddr 10.0.0.2 tcp dport 27374 meta mark 0x00000023 counter packets 0 bytes 0 accept
| add rule ip t c ip saddr 10.0.0.1 ip daddr 10.0.0.2 tcp dport 27374 meta mark 0x00000023 counter packets 0 bytes 0 accept
| netlink: Error: Could not process rule: No space left on device
Is this a bug in your patch?
Cheers, Phil
