Didn't include lvs-devel in CC of the original message, adding it now... On Tue, Sep 23, 2008 at 3:54 PM, Julius Volz <juliusv@xxxxxxxxxx> wrote: > On Tue, Sep 23, 2008 at 3:21 PM, Jan Engelhardt <jengelh@xxxxxxxxxx> wrote: >> >> On Tuesday 2008-09-23 09:13, Julius Volz wrote: >>> >>>Ok, the SYN/ACK from the backend is logged as --cstate INVALID in >>>PREROUTING and INPUT. This means that Netfilter thinks it doesn't >>>belong to any connection, although it just SNATed the SYN to the >>>backend correctly? Hmm... how can this be? >> >> That probably means skb->nfct is lost (set to NULL, which is what INVALID >> indicates) after SNAT (PREROUTING), when IPVS kicks in. > > But at PREROUTING, the skb hasn't seen any part of IPVS yet. And the > prior SNAT for the SYN packet happened in POSTROUTING after all of > IPVS was finished. Basically, there should be no IPVS involved from > the time that the SYN is correctly SNATed and the time that the > SYN/ACK is not. Please correct me if I'm missing something here... Ok, after lots of debugging, I've found the problem and got SNAT working with IPVS (in a very hacky way). The problem was that Netfilter already assigns a conntrack entry (in skb->nfct) to the CIP->VIP SYN packet in PREROUTING. This contains a tuple which contains source and destination addresses as seen by Netfilter during PREROUTING (before IPVS). IPVS then modifies the destination address without updating the associated skb->nfct entry. Later, Netfilter SNAT in POSTROUTING finds the already assigned tuple and updates it with the new source address, but the destination address changed by IPVS is never updated in this tuple. This is why this wrong conntrack entry does not match the SYN/ACK packet returning from the backend server. So as a proof of concept for IPVS SNAT, it is enough to remove the ip_vs_post_routing() function (to let Netfilter handle anything at all in POSTROUTING) and put a simple "skb->nfct = NULL;" into the IPVS NAT transmitter function, ip_vs_nat_xmit(), to let the conntrack tuple be regenerated in by Netfilter POSTROUTING. Of course, this is a memory leak, the conntrack hash entries aren't removed, etc. I guess to do this correctly, we should probably update the skb->nfct entry in IPVS while doing our manual DNAT? I realize there's a big patch integrating IPVS with conntrack (http://www.ssi.bg/~ja/nfct/) in some way, but it seems like that does much more than would be needed for SNAT. Julius -- Julius Volz - Corporate Operations - SysOps Google Switzerland GmbH - Identification No.: CH-020.4.028.116-1 -- To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
