Hello Phil,
Thank you for your reply. It is very unfortunate indeed. Here is the scenario where I thought to use a non-anonymous vmap.
Each k8s service can have 0, 1 or more associated endpoints, backends (pods providing this service). 0 endpoint already taken care of in filter prerouting hook. When there are 1 or more, proxy needs to load balance incoming connections between endpoints.I thought to create vmap per service with 1 rule per service . When an endpoint gets updated (add/deleted) which could happen anytime then the only vmap get corresponding update and my hope was that automagically load balancing will be adjusted to use updated endpoints list.
With what you explained, I am not sure if dynamic load balancing is possible at all. If numgen work only with static anonymous vmap and fixed modulus , the only way to address this dynamic nature of endpoints is to recreate service rule everytime when number of endpoints changes (recalculate modulus and entries in vmap). I suspect it is way less efficient.
What will happen to dataplane and packets in transit when the rule will be deleted and then recreated? I suspect it might result in dropped packets, could you please comment on the possible impact?
If you could suggest a better approach for the described scenario, appreciate if you share it.
Thank you
Serguei
On 2019-12-04, 5:18 AM, "n0-1@xxxxxxxxxxxxx on behalf of Phil Sutter" <n0-1@xxxxxxxxxxxxx on behalf of phil@xxxxxx> wrote:
Hi Serguei,
On Wed, Dec 04, 2019 at 12:54:05AM +0000, Serguei Bezverkhi (sbezverk) wrote:
> Nftables wiki gives this example for numgen:
>
> nft add rule nat prerouting numgen random mod 2 vmap { 0 : jump mychain1, 1 : jump mychain2 }
>
> I would like to use it but with map reference, like this:
>
> nft add rule nat prerouting numgen random mod 2 vmap @service1-endpoints
>
> Could you please confirm if it is supported? If it is what would be the type of the key in such map? I thought it would be integer, but command fails.
>
> sudo nft --debug all add map ipv4table k8s-57XVOCFNTLTR3Q27-endpoints { type integer : verdict \; }
> Error: unqualified key type integer specified in map definition
> add map ipv4table k8s-57XVOCFNTLTR3Q27-endpoints { type integer : verdict ; }
> ^^^^^^^^^^^^^^^^^^^^^^^^^^
Yes, this is sadly not possible right now. numgen type is 32bit integer,
but we don't have a type definition matching that. Type 'integer' is
unqualified regarding size, therefore unsuitable for use in map/set
definitions.
This all works when using anonymous set/map because key type is
deduced from map LHS.
We plan to support a 'typeof' keyword at some point to allow for the
same deduction from within named map/set declarations, but it needs
further work as the type info is lost on return path (when listing) so
it would create a ruleset that can't be fed back.
> The ultimate goal is to update dynamically just the map with available endpoints and loadbalance between them without touching the rule.
I don't quite understand why you need to dynamically change the
load-balancing rule: numgen modulus is fixed anyway, so the number of
elements in vmap are fixed. Maybe just jump to chains and dynamically
update those instead?
Cheers, Phil
