Re: load balancing between two chains

[sbezverk <sbezverk@xxxxxxxxx>]

Hello,

Changed kernel to 5.4.10, and switch to use "inc" instead of "random".  Now first curl works and second fails. Whenever second chain is selected to be used,  curl connection gets stuck. 

        chain k8s-nfproxy-svc-M53CN2XYVUHRQ7UB {
                numgen inc mod 2 vmap { 0 : goto k8s-nfproxy-sep-TMVEFT7EX55F4T62, 1 : goto k8s-nfproxy-sep-GTJ7BFLUOQRCGMD5 }
                counter packets 1 bytes 60 comment ""
        }

        chain k8s-nfproxy-sep-TMVEFT7EX55F4T62 {
                counter packets 1 bytes 60 comment ""
                ip saddr 57.112.0.41 meta mark set 0x00004000 comment ""
                dnat to 57.112.0.41:8080 fully-random
        }

        chain k8s-nfproxy-sep-GTJ7BFLUOQRCGMD5 {
                counter packets 0 bytes 0 comment ""
                ip saddr 57.112.0.52 meta mark set 0x00004000 comment ""
                dnat to 57.112.0.52:8989 fully-random
        }

Any debug I could enable to see where the packet goes?

Thank you
Serguei
On 2020-01-20, 12:06 PM, "Florian Westphal" <fw@xxxxxxxxx> wrote:

    sbezverk <sbezverk@xxxxxxxxx> wrote:
    > HI Phil,
    > 
    > There is no loadblancer, curl is executed from the actual node with both pods, so all traffic is local to the node.
    > 
    > As per your suggestion I modified nfproxy rules:
    > 
    >         chain k8s-nfproxy-svc-M53CN2XYVUHRQ7UB {
    >                 numgen random mod 2 vmap { 0 : goto k8s-nfproxy-sep-I7XZOUOVPIQW4IXA, 1 : goto k8s-nfproxy-sep-ZNSGEJWUBCC5QYMQ }
    >                 counter packets 3 bytes 180 comment ""
    >         }
    > 
    >         chain k8s-nfproxy-sep-ZNSGEJWUBCC5QYMQ {
    >                 counter packets 0 bytes 0 comment ""
    >                 ip saddr 57.112.0.38 meta mark set 0x00004000 comment ""
    >                 dnat to 57.112.0.38:8080 fully-random
    >         }
    > 
    >         chain k8s-nfproxy-sep-I7XZOUOVPIQW4IXA {
    >                 counter packets 1 bytes 60 comment ""
    >                 ip saddr 57.112.0.36 meta mark set 0x00004000 comment ""
    >                 dnat to 57.112.0.36:8989 fully-random
    >         }
    
    Weird, it looks like it generates 0 and something else, not 1.
    
    Works for me on x86_64 with 5.4.10 kernel:
    
    table ip test {
            chain output {
                    type filter hook output priority filter; policy accept;
                    jump k8s-nfproxy-svc-M53CN2XYVUHRQ7UB
            }
    
            chain k8s-nfproxy-svc-M53CN2XYVUHRQ7UB {
                    numgen random mod 2 vmap { 0 : goto k8s-nfproxy-sep-I7XZOUOVPIQW4IXA, 1 : goto k8s-nfproxy-sep-ZNSGEJWUBCC5QYMQ }
                    counter packets 0 bytes 0
            }
    
            chain k8s-nfproxy-sep-ZNSGEJWUBCC5QYMQ {
                    counter packets 68602 bytes 5763399
                    ip saddr 57.112.0.38 meta mark set 0x00004000 comment ""
            }
    
            chain k8s-nfproxy-sep-I7XZOUOVPIQW4IXA {
                    counter packets 69159 bytes 5809685
                    ip saddr 57.112.0.36 meta mark set 0x00004000 comment ""
            }
    }
    
    (I removed nat rules and then ran ping -f 127.0.0.1).
    
    Does it work when you use "numgen inc" instead of "numgen rand" ?