Hi,
According to api folks kube-proxy must sustain 5k or about test otherwise it will never see production environment. Implementing of numgen expression is relatively simple, thanks to "nft --debug all" once it's done, a user can use it as easily as with json __
Regarding concurrent usage, since my primary goal is kube-proxy I do not really care at this moment, as k8s cluster is not an application you co-locate in production with some other applications potentially altering host's tables. I agree firewalld might be interesting and more generic alternative, but seeing how quickly things are done in k8s, maybe it will be done by the end of 21st century __
Once I get filter chain portion in the code I will share a link to repo so you could review.
Thanks a lot for this discussion, very useful
Serguei
On 2019-11-27, 11:08 AM, "n0-1@xxxxxxxxxxxxx on behalf of Phil Sutter" <n0-1@xxxxxxxxxxxxx on behalf of phil@xxxxxx> wrote:
Hi,
On Wed, Nov 27, 2019 at 03:35:04PM +0000, Serguei Bezverkhi (sbezverk) wrote:
> No, I do not, nftableslib talks directly talk to netlink connection.
>
> nftableslib offers an API which allows create tables/chains/rules and exposes an interface which looks similar to k8s client-go. If you check https://github.com/sbezverk/nftableslib/blob/master/cmd/e2e/e2e.go
>
> It will give you a good idea how it operates.
>
> The reason for going in this direction is performance, for a relatively static applications like a firewall, json approach is great, but for applications like a kube-proxy where hundreds or even thousands of service/endpoint events happen, I do not believe json is a right approach. When I talked to api machinery folks I was given 5k events per second as a target.
So you're bypassing both libnftables and libnftnl. Those 5k events per
second are a benchmark, not an expected load, right?
While you're obviously searching for the most performance, the drawback
is complexity. Using JSON (and thereby libnftables and libnftnl as
backends) a task like utilizing numgen expression is relatively simple.
A problem you won't get rid of with the move from iptables to nftables
is concurrent use: The "let's insert our rules on top" approach to
dealing with an existing ruleset or other users is obviously not the
best one. I guess you're aiming at dedicated applications where this is
not an issue but for "general purpose" applications I guess a k8s
backend communicating with firewalld would be a good approach of
customizing host's firewall setup without stepping onto others' toes.
Back to topic, you are creating a static ruleset based on the iptables
one you got for simple comparison tests or are you already over that? If
not, I guess it would be a good basis for high level ruleset
optimization discussions.
Cheers, Phil
