The behavior is random, as you can see. Here, delete element commands
were processed ok, but there were problems during flush chain so
referenced maps couldn't be deleted.
1 # fCreateRouter 10.5.59.166
2 add element ip filter router_map { 10.5.59.166 : "1:0x508" }
3 # fCreateRouter 10.5.56.5
4 add element ip filter router_map { 10.5.56.5 : "1:0x509" }
5 # fDeleteTarget 10.1.1.55
6 delete element ip filter group_3565_prio { 10.1.1.55/32 }
7 delete element ip filter group_3565 { 10.1.1.55/32 }
8 # fDeleteTarget 10.1.1.52
9 delete element ip filter group_3565_prio { 10.1.1.52/32 }
10 delete element ip filter group_3565 { 10.1.1.52/32 }
11 # fDeleteTarget 10.1.1.39
12 delete element ip filter group_3565_prio { 10.1.1.39/32 }
13 delete element ip filter group_3565 { 10.1.1.39/32 }
14 # fDeleteTarget 10.1.1.35
15 delete element ip filter group_3565_prio { 10.1.1.35/32 }
16 delete element ip filter group_3565 { 10.1.1.35/32 }
17 # fDeleteTarget 10.1.1.49
18 delete element ip filter group_3565_prio { 10.1.1.49/32 }
19 delete element ip filter group_3565 { 10.1.1.49/32 }
20 # fDeleteTarget 10.1.1.38
21 delete element ip filter group_3565_prio { 10.1.1.38/32 }
22 delete element ip filter group_3565 { 10.1.1.38/32 }
23 # fDeleteTarget 10.1.1.41
24 delete element ip filter group_3565_prio { 10.1.1.41/32 }
25 delete element ip filter group_3565 { 10.1.1.41/32 }
26 # fDeleteTarget 10.1.1.58
27 delete element ip filter group_3565_prio { 10.1.1.58/32 }
28 delete element ip filter group_3565 { 10.1.1.58/32 }
29 # fDeleteSubnet group_3565 10.1.1.32/27
30 delete element ip filter subnet_map {10.1.1.32/27}
31 # fDeleteGroup group_3565
32 flush chain ip filter group_3565
33 delete map ip filter group_3565
34 delete map ip filter group_3565_prio
35 delete chain ip filter group_3565
36 # fDeleteTarget 10.5.3.137
37 delete element ip filter group_8426_prio { 10.5.3.137/32 }
38 delete element ip filter group_8426 { 10.5.3.137/32 }
39 # fDeleteTarget 10.5.3.136
40 delete element ip filter group_8426_prio { 10.5.3.136/32 }
41 delete element ip filter group_8426 { 10.5.3.136/32 }
42 # fDeleteTarget 10.5.3.134
43 delete element ip filter group_8426_prio { 10.5.3.134/32 }
44 delete element ip filter group_8426 { 10.5.3.134/32 }
45 # fDeleteTarget 10.5.3.135
46 delete element ip filter group_8426_prio { 10.5.3.135/32 }
47 delete element ip filter group_8426 { 10.5.3.135/32 }
48 # fDeleteTarget 10.5.3.133
49 delete element ip filter group_8426_prio { 10.5.3.133/32 }
50 delete element ip filter group_8426 { 10.5.3.133/32 }
51 # fDeleteTarget 10.5.3.131
52 delete element ip filter group_8426_prio { 10.5.3.131/32 }
53 delete element ip filter group_8426 { 10.5.3.131/32 }
54 # fDeleteTarget 10.5.3.132
55 delete element ip filter group_8426_prio { 10.5.3.132/32 }
56 delete element ip filter group_8426 { 10.5.3.132/32 }
57 # fDeleteTarget 10.5.3.130
58 delete element ip filter group_8426_prio { 10.5.3.130/32 }
59 delete element ip filter group_8426 { 10.5.3.130/32 }
60 # fDeleteSubnet group_8426 10.5.3.128/28
61 delete element ip filter subnet_map {10.5.3.128/28}
62 # fDeleteGroup group_8426
63 flush chain ip filter group_8426
64 delete map ip filter group_8426
65 delete map ip filter group_8426_prio
66 delete chain ip filter group_8426
67 # fCreateGroup group_3565
68 # fReachableGroup group_3565
69 flush chain ip filter group_3565
70 add rule ip filter group_3565 meta priority 0 ip saddr
@priority_set meta priority set ip daddr map @group_3565_prio counter
71 add rule ip filter group_3565 meta priority 0 ip daddr
@priority_set meta priority set ip saddr map @group_3565_prio counter
72 add rule ip filter group_3565 meta priority 0 meta priority set
ip daddr map @group_3565 counter
73 add rule ip filter group_3565 meta priority 0 meta priority set
ip saddr map @group_3565 counter
74 add rule ip filter group_3565 meta priority 0 counter log prefix
"group_3565 - "
75 # fCreateGroup group_8426
76 # fReachableGroup group_8426
77 flush chain ip filter group_8426 - I noticed my error here since
I'm trying to flush already deleted chain, but it demonstrates that
sometimes chains are deleted and sometimes not.
/var/spool/shaperd/nftables.nft:33:1-32: Error: Could not process
rule: Device or resource busy
delete map ip filter group_3565
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/var/spool/shaperd/nftables.nft:34:1-37: Error: Could not process
rule: Device or resource busy
delete map ip filter group_3565_prio
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/var/spool/shaperd/nftables.nft:35:1-34: Error: Could not process
rule: Device or resource busy
delete chain ip filter group_3565
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/var/spool/shaperd/nftables.nft:64:1-32: Error: Could not process
rule: Device or resource busy
delete map ip filter group_8426
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/var/spool/shaperd/nftables.nft:65:1-37: Error: Could not process
rule: Device or resource busy
delete map ip filter group_8426_prio
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/var/spool/shaperd/nftables.nft:66:1-34: Error: Could not process
rule: Device or resource busy
delete chain ip filter group_8426
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/var/spool/shaperd/nftables.nft:69:1-33: Error: Could not process
rule: No such file or directory
flush chain ip filter group_3565
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/var/spool/shaperd/nftables.nft:77:1-33: Error: Could not process
rule: No such file or directory
flush chain ip filter group_8426
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
----
S pozdravem / Best Regards
Vaclav Zindulka
On Fri, Feb 15, 2019 at 2:07 PM Václav Zindulka
<vaclav.zindulka@xxxxxxxxxx> wrote:
>
> > Works for me with nft from git:
> I've updated nftables to latest commit from git - I had it locked to
> last working state because of recent build problems. It still doesn't
> work for me from time to time. I'm attaching screenshot to show my
> problem with evidence item existed before deletion. I can provide
> whole structure including whole list of commands to create it if
> needed. Whole nftables structure contains 59250 rows.
>
> In the lower left part of the screenshot there is output of nft list
> ruleset before applying any changes. In the upper left part of screen
> there is list of commands to process and delete statements are first
> ones. They both fail. Also eighth command fails, which should process
> ok. I'm suspecting there could be some problem with flushing chain
> group_8165, because it references both maps so they can't be deleted
> but when I flush the chain by hand I can remove both maps. I've tried
> flushing other chain using libnftables and it works. In this case,
> however, it doesn't. Like it doesn't have enough time to process and
> then consequent commands fail too. In the right side area I'm checking
> existence of element in map by hand and trying to remove it.
> Unsuccessfully. Nft nor libnftables work in this case and many other
> similar cases. I'm using nft_run_cmd_from_filename function since it
> allows batch capabilities.
>
> /var/spool/shaperd/nftables.nft:2:1-61: Error: Could not process rule:
> No such file or directory
> delete element ip filter group_8165_prio { 10.143.10.96/27 }
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> /var/spool/shaperd/nftables.nft:3:1-56: Error: Could not process rule:
> No such file or directory
> delete element ip filter group_8165 { 10.143.10.96/27 }
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> /var/spool/shaperd/nftables.nft:8:1-32: Error: Could not process rule:
> Device or resource busy
> delete map ip filter group_8165
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> /var/spool/shaperd/nftables.nft:9:1-37: Error: Could not process rule:
> Device or resource busy
> delete map ip filter group_8165_prio
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> /var/spool/shaperd/nftables.nft:10:1-34: Error: Could not process
> rule: Device or resource busy
> delete chain ip filter group_8165
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> /var/spool/shaperd/nftables.nft:32:1-32: Error: Could not process
> rule: Device or resource busy
> delete map ip filter group_3564
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> /var/spool/shaperd/nftables.nft:33:1-37: Error: Could not process
> rule: Device or resource busy
> delete map ip filter group_3564_prio
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> /var/spool/shaperd/nftables.nft:34:1-34: Error: Could not process
> rule: Device or resource busy
> delete chain ip filter group_3564
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> /var/spool/shaperd/nftables.nft:36:1-62: Error: Could not process
> rule: No such file or directory
> delete element ip filter group_3569_prio { 192.168.124.5/32 }
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> /var/spool/shaperd/nftables.nft:37:1-57: Error: Could not process
> rule: No such file or directory
> delete element ip filter group_3569 { 192.168.124.5/32 }
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> /var/spool/shaperd/nftables.nft:42:1-32: Error: Could not process
> rule: Device or resource busy
> delete map ip filter group_3569
>
> commands:
> 1 # fDeleteTarget 10.143.10.96
> 2 delete element ip filter group_8165_prio { 10.143.10.96/27 }
> 3 delete element ip filter group_8165 { 10.143.10.96/27 }
> 4 # fDeleteSubnet group_8165 10.143.10.96/27
> 5 delete element ip filter subnet_map {10.143.10.96/27}
> 6 # fDeleteGroup group_8165
> 7 flush chain ip filter group_8165
> 8 delete map ip filter group_8165
> 9 delete map ip filter group_8165_prio
> 10 delete chain ip filter group_8165
>
> nft list ruleset:
> map group_8165 {
> type ipv4_addr : classid
> flags interval
> elements = { 10.143.10.96/27 : 1:d656 }
> }
>
> map group_8165_prio {
> type ipv4_addr : classid
> flags interval
> elements = { 10.143.10.96/27 : 1:d657 }
> }
>
>
> there is prototype of structure I'm maintaining, but I have thousands
> of groups in two states. With 5 rules utilizing 2 maps and with one
> rule without maps. Sometimes I'm switching between states according to
> changes in routing tables.
>
> table ip filter {
> map subnet_map {
> type ipv4_addr : verdict
> flags interval
> elements = { 10.20.255.0/25 : goto group10, 10.20.255.128/25 :
> goto group11 }
> }
>
> map router_map {
> type ipv4_addr : classid
> elements = { 10.20.0.13 : 1:2 }
> }
>
> set priority_set {
> type ipv4_addr
> flags interval
> elements = { 10.20.2.1 }
> }
>
> map group10_prio {
> type ipv4_addr : classid
> flags interval
> elements = { 10.20.255.10 : 1:fffc, 10.20.255.14 : 1:fff9,
> 10.20.255.18 : 1:fff6 }
> }
>
> map group10 {
> type ipv4_addr : classid
> flags interval
> elements = { 10.20.255.10 : 1:fffb, 10.20.255.14 : 1:fff8,
> 10.20.255.18 : 1:ffe5 }
> }
>
> chain forward {
> type filter hook forward priority filter; policy accept;
> ip saddr 10.20.255.10 meta nftrace set 1
> meta priority none ip daddr vmap @subnet_map counter packets 0 bytes 0
> meta priority none ip saddr vmap @subnet_map counter packets 0 bytes 0
> meta priority none meta priority set ip daddr map @router_map
> counter packets 0 bytes 0
> meta priority none meta priority set ip saddr map @router_map
> counter packets 0 bytes 0
> meta priority none counter packets 0 bytes 0 log prefix "forward - "
> }
>
> chain group11 {
> meta priority none meta priority set 1:b ip daddr 10.20.255.128/25
> counter packets 0 bytes 0
> meta priority none meta priority set 1:b ip saddr 10.20.255.128/25
> counter packets 0 bytes 0
> meta priority none counter packets 0 bytes 0 log prefix "group11 - "
> }
>
> chain group10 {
> meta priority none ip saddr @priority_set meta priority set ip
> daddr map @group10_prio counter packets 0 bytes 0
> meta priority none ip daddr @priority_set meta priority set ip
> saddr map @group10_prio counter packets 0 bytes 0
> meta priority none meta priority set ip daddr map @group10 counter
> packets 0 bytes 0
> meta priority none meta priority set ip saddr map @group10 counter
> packets 0 bytes 0
> meta priority none counter packets 0 bytes 0 log prefix "group10 - "
> }
> }
> ----
> S pozdravem / Best Regards
>
> Vaclav Zindulka
>
> On Thu, Feb 14, 2019 at 12:18 PM Florian Westphal <fw@xxxxxxxxx> wrote:
> >
> > Václav Zindulka <vaclav.zindulka@xxxxxxxxxx> wrote:
> > > I've discovered problem with maps in nftables. When I try to remove
> > > last element of the map I get No such file or directory error.
> >
> > Works for me with nft from git:
> >
> > # nft list ruleset
> > table ip test {
> > map group_12058 {
> > type ipv4_addr : classid
> > flags interval
> > elements = { 10.13.25.32/29 : 1:b8a1 }
> > }
> > }
> > # nft delete element test group_12058 { 10.13.25.32/29 }
> > # nft list ruleset
> > table ip test {
> > map group_12058 {
> > type ipv4_addr : classid
> > flags interval
> > }
> > }
> >
> >
