On 18-04-19, Jozsef Kadlecsik wrote:
> Hi,
>
> On Wed, 17 Apr 2019, Brett Mastbergen wrote:
>
> > On 16-04-19, Florian Westphal wrote:
> > > Brett Mastbergen <bmastbergen@xxxxxxxxxxxx> wrote:
> > > > This patch looks great. That would greatly improve the usefulness of keying
> > > > off of the conntrack id. If it gets accepted I can send a kernel patch and
> > > > an nft patch to support the ct id key.
> > >
> > > That would be really nice to have.
> > >
> > > > > > For a more in depth description of how things work I suggest reading the doc
> > > > > > in the first link I posted, but below are a few simple examples of what is
> > > > > > possible using dict expressions:
> > > > > >
> > > > > > nft add rule ip filter forward dict sessions ct id application long_string
> > > > > > NETFLIX reject
> > > > > >
> > > > > > If I were to describe that rule in plain English it would be:
> > > > > >
> > > > > > For traffic passing through the ip filter table forward hook, use the
> > > > > > conntrack id as a key to lookup an entry in the sessions table. For that
> > > > > > entry check if it has an application field set to a string value of NETFLIX,
> > > > > > if so, reject the traffic.
> > > > > >
> > > > > > How did that field get set to NETFLIX? That is up to some other entity. In
> > > > >
> > > > > Why isn't it possible to use the existing nftables set infrastructure
> > > > > for this?
> > > > >
> > > > > The nft sets store arbitrary octets/bytes as keys, so we could at least
> > > > > from kernel side store arbitrary identifiers (strings, integers etc).
> > > > >
> > > >
> > > > I wanted to use the existing set infrastructure, but I ran into two issues
> > > > wrt to what we were trying to acheive:
> > > >
> > > > 1. As far as I can tell the current sets are only set up to hold elements
> > > > of a particular data type, where as we are storing elements of many different
> > > > types in a particular dictionary.
> > >
> > > Yes, a set is only one data type (They support combining types though).
> > >
> > > > 2. sets are associated with a particular table and therefore can only be
> > > > accessed by rules in that table. If you want to associate some piece of
> > > > information with a particular connection and use it from rules in multiple
> > > > different tables, you'd have to populate it in a set in each of those tables.
> > >
> > > Yes, but thats intentional, a table forms a namespace; so
> > > crossing it is not desireable.
> > >
> > > The idea is that different entities each can manage their own tables
> > > without clashing with chain or set names of another table.
> > >
> > > Still looking at dicts code, I am trying to understand where normal nft
> > > set infra can't be used (or is too cumbersome) to best understand how we
> > > can fit dicts ideas into nftables' architecture.
> >
> > The idea of some entity managing a table and its sets,rules,maps,etc in
> > its own separate namespace makes total sense. Thats one of the cool
> > things about nftables. I think what inspired us to create data stores
> > that are more global, that can be accessed/updated from any table, is
> > conntrack data. If you think about things like the conntrack state,
> > mark, bytes, status, etc, that is meta data about a connection that can
> > be accessed from any table as long as the conntrack has been created.
> > What we call the sessions table is really just more meta data about the
> > connection that is keyed off of the conntrack id that can be
> > accessed/updated from any table. In our system we have a single entity
> > that is responsible for populating sessions data, which then allows the
> > entities managing tables to use it, or not. Just the same way any table
> > may or may not use conntrack data.
>
> Yes, it's a valid argument. And you presented the two most natural
> examples: data connected to conntrack entries and quotas (maintained in
> the "filter" table and when the subject is over its quota, drop as early
> as possible, in the "raw" table).
>
> > As Dirk and I were discussing this yesterday we thought that maybe a
> > "global" map would be close to what we are doing. Something like below:
>
> I think a "global" namespace could be the solution: sets and maps defined
> in it could be used from any table (i.e. namespace). That is the contents
> can be matched/updated/added/deleted from anywhere, but such sets/maps can
> be defined/destroyed in "global" only.
>
> nftables dictionaries are tightly coupled with their namespace because of
> the verdict/jump part, so those would not be allowed in this global
> namespace.
>
> What do you think, is that a good generalization and could properly cover
> your cases?
>
Yes, I think your description of a global namespace that sets and maps can
be defined in sounds right on and would cover our use cases as far as where
the data resides (globally) and where it is accessible from (any table).
There are a couple other considerations I'd like to throw out there:
1. Today it seems that nftables assumes that the map expressions will only
be used as the right hand expression of things like dnat and snat rules.
# from the wiki
nft add rule ip nat postrouting snat tcp dport map @porttoip
But we'd really like to be able to use map lookups on the left side of
comparisons so we can look up the additional meta data we put in a global
map and make some decision based on whether it matches:
nft add rule inet foo bar ct id map @application_map "netflix" drop
I've just about got that working locally. It seems there isn't any
architectural reason that isn't possible today, I think it was just coded
up with the assumption of being a right side expression all the time.
2. If you take a peek at our nft_dict documentation you'll notice we have
defined some new generic data types: int_type, int64_type, long_string, and
bool (I think upstream has a boolean type now though). Since we are storing
lots of artibrary meta data in our tables we didn't want to have to define
a new data type every time we decided to store some new piece of data in a
table. So instead of defining a new fixed length string subtype for
application type, we'd just define it as a long_string. And instead of
defining a new fixed length integer subtype for reputation, we'd just
define it as an int_type. Obviously, there are times when you do want
to define a new type if only certain values are valid, but we've found the
flexibility of these generic types to be really handy.
> > Create a global map from conntrack id to an application string (obviously
> > these data types and syntax are made up)
> >
> > nft add map global application_map { type ct_id: app_string ; }
> >
> > and then some table could have a rule like this:
> >
> > nft add rule inet foo bar ct id map @application_map "netflix" drop
> >
> > And then something a bit more complex: map connections to users and users
> > to quota status:
> >
> > nft add map global user_map { type ct_id: username ; }
> > nft add map global over_quota { type username: bool ; }
> > nft add rule inet foo bar ct id map @user_map map @over_quota true drop
>
> Best regards,
> Jozsef
> -
> E-mail : kadlec@xxxxxxxxxxxxxxxxx, kadlecsik.jozsef@xxxxxxxxxxxxx
> PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
> Address : Wigner Research Centre for Physics, Hungarian Academy of Sciences
> H-1525 Budapest 114, POB. 49, Hungary
