Hi!
The Netfilter project proudly presents:
nftables 0.6
This release contains many accumulated bug fixes and new features
availale up to the Linux 4.7-rc1 kernel release.
New features
============
* Rule replacement: You can replace any rule from the unique 64-bits
handle. You have to retrieve the handle from the ruleset listing.
# nft list ruleset -a
table ip filter {
chain input {
...
ct state new tcp dport ssh accept counter packets 0 bytes 0 # handle 4
}
}
Then, indicate this handle from the new rule that you want to
replace, eg.
# nft replace rule filter input handle 4 ct state new \
tcp dport { 22, 80} counter accept
* Flow table support: This provides a native replacement for the
hashlimit match in iptables. The rule below creates a 'ssh' flow table
declares a ratelimit of 10 packets per second for each source IP address:
# nft add rule filter input tcp dport 22 ct state new \
flow table ssh { ip saddr limit rate 10/second } accept
This is actually way more than hashlimit since you can use any selector
and build your own tuple of selectors through concatenations, eg.
# nft add rule filter input \
flow table acct { iif . ip saddr timeout 60s counter }
Then, if you want to list the content of the 'acct' flow table:
# nft list flow table acct
table ip filter {
flow table acct {
type iface_index . ipv4_addr
flags timeout
elements = { eth0 . 218.68.110.274 expires 3m56s : counter packets 1 bytes 98, eth0 . 180.29.103.19 expires 3m57s : counter packets 2 bytes 80, eth0 . 8.8.8.8 expires 3m44s : counter packets 1 bytes 84}
}
}
Note that this listing format is still unstable though, so don't make
tools to parse this output yet. Commands to empty flow tables and remove
specific entries are still missing.
Moreover, flow tables require a Linux kernel >= 4.3.
* New tracing infrastructure: Useful for ruleset debugging, you have
to enable tracing via:
# nft filter input tcp dport 10000 nftrace set 1
# nft filter input icmp type echo-request nftrace set 1
Then, you can monitor traces through:
# nft -nn monitor trace
That generates the following outputs:
trace id e1f5055f ip filter input packet: iif eth0 ether saddr
63:f6:4b:00:54:52 ether daddr c9:4b:a9:00:54:52 ip saddr 192.168.122.1
ip daddr 192.168.122.83 ip tos 0 ip ttl 64 ip id 32315 ip length 84
icmp type echo-request icmp code 0 icmp id 10087 icmp sequence 1
trace id e1f5055f ip filter input rule icmp type echo-request
nftrace set 1 (verdict continue)
trace id e1f5055f ip filter input verdict continue
trace id e1f5055f ip filter input
trace id 74e47ad2 ip filter input packet: iif vlan0 ether saddr
63:f6:4b:00:54:52 ether daddr c9:4b:a9:00:54:52 vlan pcp 0 vlan cfi 1
vlan id 1000 ip saddr 10.0.0.1 ip daddr 10.0.0.2 ip tos 0 ip ttl 64 ip
id 49030 ip length 84 icmp type echo-request icmp code 0 icmp id 10095
icmp sequence 1
trace id 74e47ad2 ip filter input rule icmp type echo-request
nftrace set 1 (verdict continue)
trace id 74e47ad2 ip filter input verdict continue
trace id 74e47ad2 ip filter input
trace id 3030de23 ip filter input packet: iif vlan0 ether saddr
63:f6:4b:00:54:52 ether daddr c9:4b:a9:00:54:52 vlan pcp 0 vlan cfi 1
vlan id 1000 ip saddr 10.0.0.1 ip daddr 10.0.0.2 ip tos 16 ip ttl 64
ip id 59062 ip length 60 tcp sport 55438 tcp dport 10000 tcp flags ==
syn tcp window 29200
trace id 3030de23 ip filter input rule tcp dport 10000 nftrace set
1 (verdict continue)
trace id 3030de23 ip filter input verdict continue
trace id 3030de23 ip filter input
The trace id is unique for each packet, there above you can see the
travel of this packet through the nft packet classifier.
* Ratelimiting enhancements: You can now specify ratelimits in terms
of bytes/second, eg.
# nft add rule filter forward \
limit rate 1024 mbytes/second counter accept
The rule above matches packets under the specified ratelimit. This
requires a Linux kernel >= 4.3 btw.
You can also indicate the amount of traffic that can go over the
threshold via 'burst', eg.
# nft add rule filter forward \
limit rate 1024 mbytes/second burst 10240 bytes counter accept
You may also need to match based on inverted logic, eg.
# nft add rule filter forward \
limit rate over 1024 mbytes/second log prefix "OVERLIMIT: " drop
* VLAN matching: You can match any vlan header field and combine this
with any of the existing upper layer header selectors, eg.
# nft add rule bridge filter prerouting vlan id 24 \
ip saddr 192.168.1.0/24 counter accept
* Packet duplication: When used from any of the supported layer 3 families,
this allows you to clone packets to a given destination address, eg.
duplicate all packets whose mark is 0xffff:
# nft add rule filter forward \
meta mark 0xffff dup to 172.20.0.2 counter
You can actually combine this new feature with maps:
# nft add rule filter forward meta mark 0xffff \
dup to meta mark map { \
0xffff : 172.20.0.2, \
0xeeee : 172.20.0.3 } counter
So the destination for the duplicated packet depends on the packet
mark.
You can also use this from layer 2 ingress, eg.
# nft add table netdev filter
# nft add chain netdev filter ingress { \
type filter hook ingress device eth0 priority 0\; }
# nft add rule netdev filter ingress dup to dummy0
In this case, you specify the nic that is used to transmit the
duplicated packet.
* Packet forwarding from the new ingress family, eg.
# nft add rule netdev filter ingress iif eth0 fwd to eth1
To forward packets that enter from eth0, then make them go through
eth1.
* String prefix matching through '*' (asterisk), eg.
# nft add filter forward iifname eth\* ...
This is equivalent to iptables ... -i eth+
* DSCP and ECN matching, eg.
# nft add rule filter forward ip dscp cs1 counter
# nft add rule ip6 filter forward ip6 dscp cs4 counter
# nft add rule ip filter forward ip ecn ce counter
# nft add rule ip6 filter forward ip6 ecn ce counter
* Stateless payload mangling, eg.
# nft add rule filter forward tcp dport 8080 tcp dport set 80
This rule above mangles the destination port from 8080 to 80. Beware
of interactions with conntrack, flows whose traffic is mangled must
be untracked. However, we don't support NOTRACK yet though in nft.
* Conntrack direction-based matching, eg. ct original saddr 1.2.3.4.
* Support for ICMP router advertisement and solicitation, eg.
# nft add rule ip filter input \
icmp type {router-advertisement, router-solicitation} counter accept
* Better listing support: You can perform selective listing of
objects, eg.
To display the existing declararation tables (without no content).
# nft list tables ip
table ip filter
# nft list tables ip6
table ip6 filter
# nft list tables
table ip filter
table ip6 filter
List existing chains, eg.
# nft list chains
table ip filter {
chain test1 {
}
chain test2 {
}
chain input {
type filter hook input priority 0; policy accept;
}
}
List existing set declaration, eg.
# nft list sets
table ip filter {
set libssh {
type ipv4_addr
}
}
Note the listing above shows no elements, to see the set content,
you must specify set name, eg.
# nft list set ip filter libssh
table ip filter {
set libssh {
type ipv4_addr
elements = { 163.123.166.2}
}
}
You can also list existing maps, eg.
# nft list map ip6 filter test
table ip6 filter {
map test {
type ipv6_addr : inet_service
elements = { 2001:db8::ff00:42:8329 : http}
}
}
In general, the same logic applies for every nft object, ie. generic
listing shows declarations, then if the object name is specified, the
its content is shown.
* Masquerading port range selection: Allows us to restricts the ports
that will be used in the masquerading.
# nft add rule nat postrouting ip protocol tcp masquerade to :1024-10024
* A new shell-based testsuite, to complement the existing python-based
unitary tests. Nothing fancy for users, but useful for us developers :)
Bugfixes
========
Not strictly limited to this list below, but some highlights:
* Resolve problems with prefixes in named sets, eg.
# nft add set filter blacklist { type ipv4_addr\; flags interval\; }
# nft add element filter blacklist { 192.168.1.0/24 }
# nft add element filter blacklist { 192.168.5.0/24, 192.168.7.0/24 }
# nft delete element filter blacklist { 192.168.5.0/24, 192.168.7.0/24 }
Note: this requires a couple of patches in Linux >= 4.7-rc1, I will
request submission to -stable asap.
* Resolve dynamic map evaluation problems, so this below now works:
# cat ruleset.file
table ip mangle {
map CLASS05 {
type ipv4_addr : mark
elements = { 192.168.0.10 : 0x00000001}
}
chain OUTPUT {
type route hook output priority 0; policy accept;
mark set ip saddr map @CLASS05
}
}
# nft -f ruleset.file
* Filtering based on layer 2 header selectors from the in inet family,
eg.
# nft add rule inet filter forward ether saddr 00:0f:54:0c:11:40 \
tcp dport 22 counter accept
* Fix wrong dependency handling, eg. ip protocol != tcp udp dport ssh
* Enforce ip6 proto with exthdr expression.
* Generate the correct bytecode on NAT redirection where ports are
specified, eg.
# nft add rule ip nat prerouting tcp dport 80 redirect to 1025-2048
* Printing of rule comments where misplace when listing the rule
handle, the example below shows the right output:
# nft list ruleset -a
table filter {
chain input {
...
iifname eth0 comment "test" # handle 1
}
}
* Restore matching of icmp redirect type and ct status snat and dnat.
Resources
=========
The nftables code can be obtained from:
* http://netfilter.org/projects/nftables/downloads.html
* ftp://ftp.netfilter.org/pub/nftables
* git://git.netfilter.org/nftables
To build the code, libnftnl 1.0.6 and libmnl >= 1.0.2 are required:
* http://netfilter.org/projects/libnftnl/index.html
* http://netfilter.org/projects/libmnl/index.html
Visit our wikipage for user documentation at:
* http://wiki.nftables.org
For the manpage reference, check man(8) nft.
In case of bugs and feature request, file them via:
* https://bugzilla.netfilter.org
Make sure you create no duplicates already, thanks!
Happy testing!
Arturo Borrero (27):
tests/: rearrange tests directory
tests/: add shell test-suite
tests/shell: add maps tests cases
tests/shell: add test case for cache bug
tests/shell: add tests for handles and comments
rule: don't list anonymous sets
rule: delete extra space in sets printing
tests/shell: add first `nft -f' tests
tests/listing: add some listing tests
tests/shell: unload modules between tests
tests/shell/run-tests.sh: tune kernel cleanup
tests/shell: add chain validations tests
rule: don't print trailing statement whitespace
tests/shell: add new testcases for commit/rollback
tests/shell: add some tests for network namespaces
evaluate: improve rule management checks
test: shell: also unload NAT modules
tests/shell: add testcases for Netfilter bug #965
tests/shell: delete tempfile failover in testcases
tests: shell: add testcases for named sets with intervals
tests: py: allow to run tests with other nft binaries
dist: include tests/ directory and files in tarball
evaluate: check for NULL datatype in rhs in lookup expr
tests/shell: add testcase for 'nft -f' load with actions
tests/shell: add testcase to catch segfault if invalid syntax was used
tests/shell/run-tests.sh: execute tests in sorted order
tests/shell/run-tests.sh: print hint about testcase being executed
Arturo Borrero Gonzalez (1):
rule: fix printing of rule comments
Carlos Falgueras García (6):
src: Add command "replace" for rules
rule: Use libnftnl user data TLV infrastructure
netlink_linearize: do not duplicate user data when linearizing user data
set_elem: Use libnftnl/udata to store set element comment
parser: Consolidate comment production
parser: cap comment length to 128 bytes
Florian Westphal (54):
tests: don't depend on set element order
nft: allow stacking vlan header on top of ethernet
payload: disable payload merge if offsets are not on byte boundary.
src: netlink_linearize: handle sub-byte lengths
src: netlink: don't truncate set key lengths
nft: fill in doff and fix ihl/version template entries
netlink: cmp: shift rhs constant if lhs offset doesn't start on byte boundary
tests: add tests for ip version/hdrlength/tcp doff
nft: support listing expressions that use non-byte header fields
tests: vlan tests
vlan: make != tests work
expression: provide clone operation for set element ops
src: allow filtering on L2 header in inet family
tests: add tests matching on ether saddr for inet, bridge, ip, ip6
rule: don't reorder protocol payload expressions when merging
tests: add test cases for ethernet header matching
tests: add inet test for ip/ether concatenation
tests: regression: fix arp.t expected payload
doc: update meta and ct expression keyword lists
ct: add support for directional keys
netlink: don't handle lhs zero-length expression as concat type
netlink: only drop mask if it matches left known-size operand
src: ct: make ct l3proto work
tests: add ct tests for ip family
nft: swap key and direction in ct_dir syntax
ct: add packet/byte counter support
netlink_linearize: use u64 conversion for 64bit quantities
ct regression tests for bytes, packets
tests: ct: remove BUG cases that work with current master
doc: update ct expression
netlink: move binop postprocess to extra function
tests: add two map test cases
netlink: do binop postprocessing also for map lookups
netlink_delinearize: only remove protocol if equal cmp is used
meta: fix error checks in tc handle parser
examples: use current type names
evaluate: reject set references in set elements
evaluate: enforce ip6 proto with exthdr expression
netlink: split generic part of netlink_gen_payload_mask into helper
netlink: add and use netlink_gen_exthdr_mask
payload: move payload_gen_dependency generic part to helper
exthdr: generate dependencies for inet/bridge/netdev family
tests: add/fix inet+exthdr tests
exthdr: remove implicit dependencies
exthdr: store offset for later use
netlink_delinearize: prepare binop_postprocess for exthdr demux
netlink_delinearize: handle extension header templates with odd sizes
tests: frag: enable more tests
netlink_delinearize: fix bogus offset w exthdr expressions
nft-test: don't zap remainder of rule after handling a set
netlink_delinarize: shift constant for ranges too
tests: frag: enable more tests
payload: only merge if adjacent and combined size fits into a register
netlink_delinerize: don't use meta_match_postprocess for ct pp
Laura Garcia Liebana (3):
proto: Add router advertisement and solicitation icmp types
doc: fix compression parameter index
doc: fix old parameters and update datatypes
Liping Zhang (4):
evaluate: fix crash if we add an error format rule
meta: fix endianness in priority
meta: fix a format error display when we set priority to root or none
parser: fix crash if we add a chain with an error chain type
Magnus Öberg (1):
build: include/mini-gmp.h is not included at "make dist"
Pablo M. Bermudo Garay (16):
tests: regression: homogenize indentation style
tests: regression: allow to run tests from anywhere
tests: add *.got files to .gitignore
tests: remove useless logic
tests: fix crash when rule test is malformed
tests/py: remove unused variables
tests/py: fix style
tests/py: simplify use of globals
tests/py: convert chains and tables to objects
tests/py: modify supported test file syntax
tests/py: update test files syntax
rule: add 'list flow tables' support
rule: add support for display flow tables content
src: add 'list maps' support
src: add support for display maps content
evaluate: fix "list set" unexpected behaviour
Pablo Neira Ayuso (96):
src: add per-bytes limit
src: add burst parameter to limit
tests: limit: extend them to validate new bytes/second and burst parameters
rule: filter out tables depending on family
parser: show all tables via list tables with no family
src: add dup statement support
tests: add tests for dup
rule: display table when listing sets
src: add `list chains' command
rule: display table when listing one set
evaluate: check if set exists before listing it
rule: `list sets' only displays declaration, not definition
rule: rework list chain
parser_bison: show all sets via list sets with no family
evaluate: check if table and chain exists when adding rules
netlink_linearize: factor out prefix generation
evaluate: fix mapping evaluation
src: add interface wildcard matching
evaluate: fix string matching on big endian
netlink_delinearize: fix use-after-free
tests: vlan pcp and cfi are located in the first byte
src: fix sub-byte protocol header definitions
netlink_delinearize: postprocess expression before range merge
netlink_delinearize: add previous statement to rule_pp_ctx
src: add new netdev protocol description
tests: py: check set value from selector and map
parser: restrict relational rhs expression recursion
parser: add redirect constant to rhs_expr rule
parser: get rid of multiton_expr from lhs relational expression
parser: rename multiton_expr to multiton_rhs_expr
parser: restore bitwise operations from the rhs of relational expressions
parser_bison: initializer_expr must use rhs_expr
tests/py: don't test log statement from protocol match
tests/py: test udp from ip and ip6 families
tests/py: netdev family with ingress chain
src: support limit rate over value
src: add dup statement for netdev
src: add fwd statement for netdev
tests/py: test port ranges and maps for redirect
evaluate: resolve_protocol_conflict() should return int
evaluate: move inet/netdev protocol context supersede logic to supersede_dep()
evaluate: check if we have to resolve a conflict in first place
evaluate: don't adjust offset from resolve_protocol_conflict()
evaluate: only try to replace dummy protocol from link-layer context
evaluate: assert on invalid base in resolve_protocol_conflict()
evaluate: wrap protocol context debunk into function
evaluate: generate ether type payload after meta iiftype
proto: proto_dev_type() returns interface type for base protocols too
src: annotate follow up dependency just after killing another
tests/py: test vlan on ingress
netlink_delinearize: prune implicit binop before payload_match_postprocess()
proto: use parameter-problem for icmpv6 type
tests/py: extend masquerade to cover ports too
rule: simplify ("rule: delete extra space in sets printing")
parser: remove 'reset' as reserve keyword
tests/py: enable tests for dccp types
parser_bison: allow 'snat' and 'dnat' keywords from the right-hand side
tests/py: add tests for router-advertisement and router-solicitation icmp types
src: revisit cache population logic
evaluate: use table_lookup_global() from expr_evaluate_symbol()
parser_bison: simplify hook_spec rule
parser_bison: duplicate string returned by chain_type_name_lookup()
parser_bison: release parsed type and hook name strings
src: store parser location for handle and position specifiers
segtree: perform stricter expression type validation from expr_value()
segtree: clone full expression from interval_map_decompose()
segtree: handle adjacent interval nodes from expr_value_cmp()
segtree: explicit initialization via set_to_intervals()
rule: support for incremental set interval element updates
segtree: special handling for the first non-matching segment
evaluate: bail out on prefix or range to non-interval set
segtree: set expr->len for prefix expression from interval_map_decompose()
segtree: add expr_to_intervals()
segtree: rename set expression set_to_segtree()
segtree: add interval overlap detection for dynamic updates
tests/py: add more interval tests for anonymous sets
tests/py: explicitly indication of set type and flags from test definitions
tests/py: add interval tests
evaluate: transfer right shifts to range side
evaluate: transfer right shifts to set reference side
src: move payload sub-byte matching to the evaluation step
evaluate: handle payload matching split in two bytes
proto: update IPv6 flowlabel offset and length according to RFC2460
proto: remove priority field definition from IPv6 header
src: add dscp support
src: add ecn support
tests/py: add missing netdev ip dscp payload tests
tests/py: fix fragment-offset field
tests/py: fix payload of dccp type in set elements
netlink: several function constifications
src: declare interval_map_decompose() from header file
tests/py: update for changed set name in payload
parser_bison: update flow table syntax
include: constify nlexpr field in location structure
tests/py: add tests for frag more-fragments and frag reserved2
Bump version to v0.6
Patrick McHardy (17):
rule: move comment out of handle
proto: add checksum key information to struct proto_desc
payload: add payload statement
proto: fix arpop symbol table endianess
netlink: fix up indentation damage
payload: fix stacked headers protocol context tracking
nft: resync kernel header files
payload: move payload depedency tracking to payload.c
payload: add payload_is_stacked()
proto: add protocol header fields filter and ordering for packet decoding
nft monitor [ trace ]
evaluate: transfer right shifts to constant side
set: allow non-constant implicit set declarations
set: explicitly supply name to implicit set declarations
netlink_delinearize: support parsing statements not contained within a rule
stmt: support generating stateful statements outside of rule context
src: add flow statement
Piyush Pangtey (3):
doc: nft: Fixed a typo and added/changed punctuation
nft: Modified punctuation used in nft's show_help
rule: Remove memory leak
Shivani Bhardwaj (5):
src: datatype: Modify symbol table for icmpv6 packet types
ip6: Add tests for icmpv6 packet types
src: netlink_linearize: Fix bug for redirect target
src: Add support for masquerade port selection
src: evaluate: Show error for fanout without balance
