Part one:
I want to have an IP for each of the services, mail and http, on each ISP, so that is DSL is down I can use cable, and vice-versa. I will do NAT in the firewall, and forward the packets to the actual server. Eventually the servers will move to a DMZ after the other stuff settles down.
The problem is that a packet can come from any IP outside, and when the reply packet is sent out, it may go out either NIC. And that's the root of the problem, getting the source IP to match the NIC. I've added rules to the mangle table to MARK the packets, that just doesn't seem to work reliably.
I want very much to do this without patching the kernel, I have two patches which seem to solve the problem on other systems, but maintaining a patched kernel long term is really undesirable, and makes it hard to turn over the job in the future.
All I want to do is send packets out the interface which matches the source IP, and I don't think there's any reasonable way to get there without patches or BSD.
Yes, I know about the lartc docs, nano.txt and several other things. The problem is that the marks don't reliably WORK, routing by destination IP is being used in some cases (but not all, which is really odd).
-- bill davidsen <davidsen@xxxxxxx> CTO TMR Associates, Inc Doing interesting things with small computers since 1979
