Hi again, I did some further experiments on this and found a workable solution for now. Details below ... On søn, 2016-09-18 at 15:19 +0200, Anders K. Pedersen | Cohaesio wrote: > I'm trying to use nftables to filter traffic from known bogon > addresses > based on the list available at > http://www.team-cymru.org/Services/Bogons/fullbogons-ipv4.txt ;. > > Loading this list initially works fine, but I also want to update it > regularly (preferably without flushing the entire nftables rule set, > which would clear all traffic counters etc.). Since there is no flush > operation for sets, I decided to delete all the members of the set, > but > this doesn't work as expected. > > I use the following shell code convert the input file to nftables > syntax: > > echo "add set ip filter bogons { type ipv4_addr; flags interval; }" > >nft.rules > echo -n "add element ip filter bogons { " >>nft.rules > I=0 > egrep -v '^(#|$)' fullbogons-ipv4.txt | > while read CIDR ; do > if [[ ${I} = 0 ]]; then > echo -n "${CIDR}" > else > echo -n ", ${CIDR}" > fi > I=1 > done >>nft.rules > echo " }" >>nft.rules > > Loading it initially with 'nft -f nft.rules' works fine. I then use > the > following shell code to try to clear it again: > > nft list set ip filter bogons | > egrep "^[[:space:]]+elements =" | > sed -e "s/.*{/delete element ip filter bogons {/" >nft.remove > nft -f nft.remove > > When I run this for the first time, nft doesn't give any output, > which > suggests that it succeeded, but only ~40% of the elements were > actually > removed. When I run the removal code again, I get no output and no > further elements were removed. > > I noticed that the nft exit code was 1 and with strace I found that > the > netlink socket got a reply with msg_flags=MSG_TRUNC just before > exiting. By adding debug output to error paths, I traced this back to > the mnl_cb_run() call in mnl_batch_talk(). By increasing the size > of rcv_buf to 32*MNL_SOCKET_BUFFER_SIZE (less may have been enough, > but > 2* wasn't) I got past that error, and nft gave the following error > message: > > nft.remove:1:1-33348: Error: Could not process rule: No such file or > directory > > I believe it is trying to tell me that the elements, I'm trying to > remove, are not present in the set, but "nft list set ip filter > bogons" > tells me that they are. After various tests, I found that if I limit the number of elements to remove to 1000 per 'delete element' statement it works. The exact breaking point with my test data was 1638 elements, but I suspect that may depend on the data used, so I settled on a lower number. In case others run into a similar problem the shell code I use for this is: nft list set ip filter bogons | perl -ne 'if (/^\s+elements =/) { s/.*\{/\{/; s/((?:,[^,]+){1000}),/$1\}\n\{/g; s/\{/delete element ip filter bogons \{/g; print; }' >nft.remove After I got this working I tried to combine the 'delete element' statements with a fresh 'add element' statement in the same file, so that I could do the set update in one transaction, but that didn't work, so for now I have to delete all the elements in one transaction and then add new elements in a second transaction, which leaves the bogon filtering ineffective for a short while every time I update the set. It would be nice if nft supported a 'flush set' operation combined with 'add element' in a single transaction, or perhaps something similar to 'ipset swap'. > I've tried to figure out what's causing this problem, but my > understanding of the netlink code isn't sufficient to make progress, > so any help would be much appreciated. The netlink code is still a bit of a mystery to me, so I didn't get far, while trying to debug this, but I may try again in the future. > My environment has Linux 4.7.4 plus the patch from > http://www.spinics.net/lists/netdev/msg389490.html ;, libmnl-1.0.4, > and current git versions of libnftnl and nftables. Regards, Anders K. Pedersen��.n��������+%������w��{.n����z�����n�r������&��z�ޗ�zf���h���~����������_��+v���)ߣ�
