On 11-04-19, Florian Westphal wrote: > Brett Mastbergen <bmastbergen@xxxxxxxxxxxx> wrote: > > My name is Brett Mastbergen. My colleage Dirk Morris and I have been > > working on some nftables functionality that we think is kind of cool so we > > figured we'd post it to the list to see if anyone had any thoughts or > > feedback on what we are doing. > > > > Below is the summary of nft_dict from our documentation which can be found > > here: > > https://github.com/untangle/nft_dict/blob/master/docs/dict.rst > > > > The nft_dict, short for netfilter dictionary, module provides a nft rule > > mechanism do table lookups on environment metadata that is not present in the > > packet and not contained within the rule. In a firewall you often wish to > > block or manipulate packets based on things not immediately evident in the > > packet, but things that can often be calculated via other mechanisms. > > > > nft_dict provides the ability to create dictionaries (lookup tables) stored > > within kernel space for fast lookups. These tables can be maintained by > > userspace applications where it is more convenient to calculate various > > network metadata. In our case, we have a userspace daemon (packetd) which > > listens to various packets with nfqueue and builds various dictionaries of > > metadata based on the traffic it sees and the information it gathers. > > > > nft_dict is a kernel module providing the kernel support for the "dict" > > expression. Its source can be found at the link below: > > > > https://github.com/untangle/nft_dict > > > > Additionally, userspace patches are required in order to use the > > "dict" expression from within an nft rule. These currently live here: > > > > https://github.com/untangle/mfw_openwrt/blob/openwrt-18.06/libnftnl/patches/999-libnftnl-Add-dict-support.patch > > https://github.com/untangle/mfw_openwrt/blob/openwrt-18.06/nftables/patches/999-nftables-Add-dict.patch > > > > These patches, along with the nft_dict kernel module, also add support for an > > "id" key to the "ct" match expression. The "ct id" expression simply returns > > the conntrack id of the conntrack. This is the same conntrack id you see if > > you run 'conntrack -L --output=id'. While not strictly required, the "ct id" > > expression is extremely useful as a key expression into a dict table for > > matching entries to a particular conntrack. > > > The correct place to implement > > the "id" ct key is in the existing nft_ct module, but for now its implemented > > in the nft_dict module. > > I would welcome adding support for this to nft_ct.c directly (i.e., get > support for a unique per-ct id) once this patch is accepted: > > https://patchwork.ozlabs.org/patch/1072650/ > > I added nf_ct_get_id() to the conntrack core so it can be called from > nft_ct later on. This patch looks great. That would greatly improve the usefulness of keying off of the conntrack id. If it gets accepted I can send a kernel patch and an nft patch to support the ct id key. > > > For a more in depth description of how things work I suggest reading the doc > > in the first link I posted, but below are a few simple examples of what is > > possible using dict expressions: > > > > nft add rule ip filter forward dict sessions ct id application long_string > > NETFLIX reject > > > > If I were to describe that rule in plain English it would be: > > > > For traffic passing through the ip filter table forward hook, use the > > conntrack id as a key to lookup an entry in the sessions table. For that > > entry check if it has an application field set to a string value of NETFLIX, > > if so, reject the traffic. > > > > How did that field get set to NETFLIX? That is up to some other entity. In > > Why isn't it possible to use the existing nftables set infrastructure > for this? > > The nft sets store arbitrary octets/bytes as keys, so we could at least > from kernel side store arbitrary identifiers (strings, integers etc). > I wanted to use the existing set infrastructure, but I ran into two issues wrt to what we were trying to acheive: 1. As far as I can tell the current sets are only set up to hold elements of a particular data type, where as we are storing elements of many different types in a particular dictionary. 2. sets are associated with a particular table and therefore can only be accessed by rules in that table. If you want to associate some piece of information with a particular connection and use it from rules in multiple different tables, you'd have to populate it in a set in each of those tables. While thats certainly doable, the dictionaries we create are "global", that is, rules from any table can access them and entries can be added and removed from a single place. > > nft add rule ip filter forward tcp dport 80 dict hosts ip saddr \ > > captive-portal-authenticated bool false dnat to 127.0.0.1:80 > > nft supports something like > > nft add rule ip filter forward tcp dport 80 ip saddr != @captive_authenticated ... > > and allows external entity to push ip addresses to the set > @captive_authenticated. For sure, sets can do that. That maybe isn't the best example of the dict's usefulness. :) > > I'll have a look at doc and code later today. Cool. I really appreciate you taking the time.
