dict: A netfilter expression for dictionary lookups

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



My name is Brett Mastbergen.  My colleage Dirk Morris and I have been
working on some nftables functionality that we think is kind of cool so we
figured we'd post it to the list to see if anyone had any thoughts or
feedback on what we are doing.

Below is the summary of nft_dict from our documentation which can be found 
here:
https://github.com/untangle/nft_dict/blob/master/docs/dict.rst

The nft_dict, short for netfilter dictionary, module provides a nft rule 
mechanism do table lookups on environment metadata that is not present in the 
packet and not contained within the rule. In a firewall you often wish to 
block or manipulate packets based on things not immediately evident in the
packet, but things that can often be calculated via other mechanisms.

nft_dict provides the ability to create dictionaries (lookup tables) stored 
within kernel space for fast lookups. These tables can be maintained by 
userspace applications where it is more convenient to calculate various 
network metadata. In our case, we have a userspace daemon (packetd) which
listens to various packets with nfqueue and builds various dictionaries of
metadata based on the traffic it sees and the information it gathers.

nft_dict is a kernel module providing the kernel support for the "dict" 
expression.  Its source can be found at the link below:

https://github.com/untangle/nft_dict

Additionally, userspace patches are required in order to use the 
"dict" expression from within an nft rule. These currently live here:

https://github.com/untangle/mfw_openwrt/blob/openwrt-18.06/libnftnl/patches/999-libnftnl-Add-dict-support.patch 
https://github.com/untangle/mfw_openwrt/blob/openwrt-18.06/nftables/patches/999-nftables-Add-dict.patch

These patches, along with the nft_dict kernel module, also add support for an 
"id" key to the "ct" match expression. The "ct id" expression simply returns 
the conntrack id of the conntrack. This is the same conntrack id you see if 
you run 'conntrack -L --output=id'. While not strictly required, the "ct id" 
expression is extremely useful as a key expression into a dict table for 
matching entries to a particular conntrack. The correct place to implement 
the "id" ct key is in the existing nft_ct module, but for now its implemented
in the nft_dict module.

For a more in depth description of how things work I suggest reading the doc 
in the first link I posted, but below are a few simple examples of what is 
possible using dict expressions:

nft add rule ip filter forward dict sessions ct id application long_string 
NETFLIX reject 

If I were to describe that rule in plain English it would be:

For traffic passing through the ip filter table forward hook, use the 
conntrack id as a key to lookup an entry in the sessions table.  For that
entry check if it has an application field set to a string value of NETFLIX,
if so, reject the traffic.

How did that field get set to NETFLIX?  That is up to some other entity.  In 
our case it is a userspace daemon that is inspecting traffic via nfqueue.
All of the metadata generated for the stream of traffic can be written into
the sessions table such that these dict rules can act upon it.  The userspace
daemon doesn't make determinations about how traffic should be acted upon
(accept, reject, drop, route, count, etc), it just populates the information
in the table to let the nft rules act upon the traffic.  This means the
userspace daemon only has to listen to traffic of a particular stream for as
long as it needs to determine all of the target metadata.  That particular
traffic stream can then bypass the nfqueue hook but can continue to be acted
upon using the information in the sessions table.

Here is another example:

nft add rule ip filter forward tcp dport 80 dict hosts ip saddr \
captive-portal-authenticated bool false dnat to 127.0.0.1:80

Here we are using the packet source ip address as the key into a hosts table
that (among other things) lets us know if this host has been authenticated via
captive portal.  If it hasn't, we send its web traffic to the local webserver.

The next example uses a dict expresssion as the key to another dict expression:

nft add rule ip filter forward dict users dict sessions ct id username \
long_string quota-exceeded bool true reject

This rule uses the conntrack id to look up the username associated with the
session, and then uses that username as the key into the users table so it
can check if this user has exceeded its quota.

If any of that sounds interesting, have a look at the code and docs.  Let us
know if you have questions or thoughts.

Thanks,
Brett




[Index of Archives]     [Netfitler Users]     [Berkeley Packet Filter]     [LARTC]     [Bugtraq]     [Yosemite Forum]

  Powered by Linux