Florian Westphal <fw@xxxxxxxxx> wrote:
> Jann Haber <jann.haber@xxxxxxxxxx> wrote:
> > We are currently running on Debian stretch with the kernel from the
> > backports:
> > jannh@nat-2:~$ uname -a
> > Linux nat-2 4.18.0-0.bpo.3-amd64 #1 SMP Debian 4.18.20-2~bpo9+1
> > (2018-12-08) x86_64 GNU/Linux
> >
> > We aim to update to buster soon, which is a 4.19.12 at the moment. Maybe
> > that would solve the problem at hand.
>
> Ok, I will check with a 4.18 one tomorrow to make sure this issue
> is really fixed (or if i made a mistake with the reproducer).
Bad news, this works for me too.
Did I misunderstand?
This is what I tried:
table inet filter {
set test {
type ipv4_addr
flags interval
elements = { 127.0.0.3, 192.168.7.0/24 }
}
chain input {
type filter hook input priority filter; policy accept;
iif "lo" ip daddr @test counter packets 102 bytes 8568
iif "eth0" ip daddr @test counter packets 365 bytes 24212
}
}
The counter for 'lo' was created by:
ping -c 1 192.168.7.10 (eth0 ip, so lo is involved: counter increases to two packets)
ping -c 100 127.0.0.1 (counter does not change)
ping -c 100 127.0.0.3 (counter increases by 100).
This is the expected behaviour.
Can you check if the ruleset above is enough to reproduce the bug on your end?
If so, then its a bug that is already fixed. I can check which patch fixes this
problem but I'm reluctant to investigate before i know the test case above is
failing for you.
> > >> - counters: In our iptables setup, we use the counters to count the
> > >> traffic of our users. We want to do the same in nftables. We therefore
> > >> created a bunch of named counters (it's about 22k of them) and a map
> > >> mapping a certain subnet/ip to the counter name. When we load the rules
> > >> with "nft -f", there seems to be a delay of some seconds, where no more
> > >> packets are processed. Since we do this frequently, these are frequent
> > >> outages of our entire network and this is unacceptable for us. When we
> > >> comment the counters and the map, the delay is gone.
> > >
> > > Could I ask why you need to re-load everything often?
> >
> > Well if we had just static counters, I agree that we should better only
> > list/reset the counters to count the traffic. However, when new members
> > register, we need to add some rules to our ruleset. Also our members can
> > add a DNAT on our webpage to our database. The data from there is then
> > translated to the nftables ruleset. A python script creates new ruleset
> > every 15 minutes. It atomically reads the counters, flushes the ruleset
> > and adds all the new rules (via nft -f).
>
> I see. nft has rule-level atomicity, so its possible to add only
> whats needed and leave rest alone rather than full replacement, but I
> understand that it might be harder to write scripts that only generate a
> delta ruleset/incremental update.
If you use python, there is libnftables that can be used from python
so no need to use 'nft' frontend.
See tests/py/nft-test.py in the nftables source for a script that uses this.
(Just a remark, no need to do this of course).
> > > I will get to it next week, adding 20k named counters should not be a
> > > problem.
> >
> > I will have to clear with our data privacy officer, maybe I can make our
> > full ruleset available to you for debugging :)
>
> If its really just caused by the named counters it should be easy to
> reproduce. I can get to it on monday, I need to finish something else
> first.
No luck either, unfortunately.
I hacked up this silly script to generate pseudo-ruleset:
-8<---8<---
echo add chain inet filter INPUT '{ type filter hook input priority 0; policy accept; }'
echo add map inet filter inipvnrmap '{ type ipv4_addr : counter; flags interval;}'
echo add map inet filter outipvnrmap '{ type ipv4_addr : counter; flags interval;}'
echo add rule inet filter INPUT counter name ip daddr map @inipvnrmap
echo add rule inet filter INPUT counter name ip saddr map @outipvnrmap
i=0
j=0
addrcount=0
while [ $i -lt 255 ]; do
while [ $j -lt 255 ]; do
iname=$(printf %s%x%x IN4 $i $j)
oname=$(printf %s%x%x OUT4 $i $j)
net="10.$i.$j.0/28"
echo add counter inet filter "$iname"
echo add counter inet filter "$oname"
echo add element inet filter inipvnrmap { $net : "$iname" }
echo add element inet filter outipvnrmap { $net : "$oname" }
j=$((j+1))
addrcount=$((addrcount+1))
done
i=$((i+1))
j=0
if [ $addrcount -gt 20000 ];then
exit 0
fi
done
-8<---8<---
nft -f generated.txt
... takes a long time to load, but a 'ping' to the vm where i am restoring
the ruleset shows no hiccups.
If you have any hint as to what I have to change to make the pings fail
intermittedly please let me know.
As for the 'long time to load', I should have patches ready to address this
in the next few days.
