Hello Phil, Please see below the list of nftables rules the code generate to mimic only filter chain portion of kube proxy. Here is the location of code programming these rules. https://github.com/sbezverk/nftableslib-samples/blob/master/proxy/mimic-filter/mimic-filter.go Most of rules are static, will be programed just once when proxy comes up, with the exception is 2 rules in k8s-filter-services chain. The reference to the list of ports can change. Ideally it would be great to express these two rules with a single rule and a vmap, where the key must be service's ip AND service port, as it is possible to have a single service IP that can be associated with several ports and some of these ports might have an endpoint and some do not. So far I could not figure it out. Appreciate your thought/suggestions/critics. If you could file an issue for anything you feel needs to be discussed, that would be great. sudo nft list table ipv4table table ip ipv4table { set svc1-no-endpoints { type inet_service elements = { 8989 } } chain filter-input { type filter hook input priority filter; policy accept; ct state new jump k8s-filter-services jump k8s-filter-firewall } chain filter-output { type filter hook output priority filter; policy accept; ct state new jump k8s-filter-services jump k8s-filter-firewall } chain filter-forward { type filter hook forward priority filter; policy accept; jump k8s-filter-forward ct state new jump k8s-filter-services } chain k8s-filter-ext-services { } chain k8s-filter-firewall { meta mark 0x00008000 drop } chain k8s-filter-services { ip daddr 192.168.80.104 tcp dport @svc1-no-endpoints reject with icmp type host-unreachable ip daddr 57.131.151.19 tcp dport @svc1-no-endpoints reject with icmp type host-unreachable } chain k8s-filter-forward { ct state invalid drop meta mark 0x00004000 accept ip saddr 57.112.0.0/12 ct state established,related accept ip daddr 57.112.0.0/12 ct state established,related accept } } Thank you Serguei On 2019-11-27, 12:22 PM, "n0-1@xxxxxxxxxxxxx on behalf of Phil Sutter" <n0-1@xxxxxxxxxxxxx on behalf of phil@xxxxxx> wrote: Hi, On Wed, Nov 27, 2019 at 04:50:56PM +0000, Serguei Bezverkhi (sbezverk) wrote: > According to api folks kube-proxy must sustain 5k or about test otherwise it will never see production environment. Implementing of numgen expression is relatively simple, thanks to "nft --debug all" once it's done, a user can use it as easily as with json __ > > Regarding concurrent usage, since my primary goal is kube-proxy I do not really care at this moment, as k8s cluster is not an application you co-locate in production with some other applications potentially altering host's tables. I agree firewalld might be interesting and more generic alternative, but seeing how quickly things are done in k8s, maybe it will be done by the end of 21st century __ I agree, in dedicated setup there's no need for compromises. I guess if you manage to reduce ruleset changes to mere set element modifications, you could outperform iptables in that regard. Run-time performance of the resulting ruleset will obviously benefit from set/map use as there are much fewer rules to traverse for each packet. > Once I get filter chain portion in the code I will share a link to repo so you could review. Thanks! I'm also interested in seeing whether there are any inconveniences due to nftables limitations. Maybe some problems are easier solved on kernel-side. Cheers, Phil
