Just a few questions question , as this makes me unsure as to why you want this setup .
1 . What is the "BOX" nat1 , just a plain VM ( virtual machine )
2. Why are your internal chassis network not just a plain BRIDGE , is there any reason for using a VLAN ?
And the most obvious question
3. Why do you not want the SNMP request with external source IP to leave the HOSTS on internal then loop around via nat1 back ?
( if it is going out externally anyway , why not just let it go the normal shortest/fastest way )
So to some stuff that I would suggest could cause the issue , but I am not sure since I do not have the complete picture .
Normally a host would choose outgoing interface based on your local routes , so if "hostA" sends an SNMP where targets are on the inside
and then use the IP on that same interface and send to whatever "router" you have in the routing table on that same interface .
Now for your scenario , as I understand it ( and there are some assumptions here , I apologies ):
STEP 1 : Your client software will get a request you made , with a destination IP it will send to outside your own 2 networks .
STEP 2 : Since a packet needs to be generated from this part of the client softtware , it "somehow" determines what interface it should use based on the routing of table "main"
( As it is not aware of the iptable MANGLE rules it cannot choose other ).
STEP 3 : Packet is created and sent through the IP STACK and reaches MANGLE table , already using the SOURCE IP of interface the client chose prior based on table "main" outgoing interface
STEP 4 : Packet is marked so that later on the routing will change for this packet , but it does not MANLGE the SOURCE IP .
STEP 5 : Packet is now routed the opposite interface with a spoofed SOURCE IP , not mathcing the subnet it is comming from even if routing allows for the destination in RP_FILTER your SOURCE IP is only allowed on the originating interface ( which is the external side of your host[ABC] ).
IF my undersanding of host[ABC] routing is correct , this is the "better" way of doing things ( again apoligies if I made some incorrect understanding and/or assumptions )
If you have 2 interfaces , and you have ROUTING disabled you should be able to do 2 things ( huge assumption , not tested ).
1. You should be able to have 2 default routes in each interface subnet
2. You host[ABC] should route packets based on origin interface
IF my assumption is incorrect , it would require 2 routing istances - but then the effect should still be the same as above if my understanding is correct .
Best regards
André Paulsberg-Csibi
Senior Network Engineer
IBM Services AS
-----Original Message-----
From: netfilter-owner@xxxxxxxxxxxxxxx [mailto:netfilter-owner@xxxxxxxxxxxxxxx] On Behalf Of FAIR, ED
Sent: Monday, December 11, 2017 10:07 PM
To: Robert White <rwhite@xxxxxxxxx>; netfilter@xxxxxxxxxxxxxxx
Subject: RE: SNMP mangling anybody?
Robert, thanks again for your thoughtful reply. I am only using the net-snmp command-line utilities as a lingua franca. In reality, my SNMP application does not use the net-snmp libraries and thus cannot use clientaddr (and it has no similar functionality to net-snmp's clientaddr).
I'm still looking in the code to find where/why rp_filter is deciding to drop the SNMP replies by default. Any insight how I might trace this?
I did make a typo in my ASCII drawing - in the line 3 comment - my profuse apologies. I incorrectly showed the prefix as 192.168.0.0/24, I should've shown as 192.168.168.0 - corrected below. I also updated the comment in line 6 to be more descriptive.
I am indeed using VLAN 2 as an internal network within the local chassis only; the idea is that one blade in the chassis (nat1) functions as a NAT translator; if any other blade in the chassis (hostA, B, or C... ) wants it's outbound traffic translated it can use nat1 for its next hop. And these two points are most important: 1) I don't want any outbound traffic from hostA, B, C translated by nat1 except their outbound SNMP requests; 2) I do not want any inbound traffic to hostA, B, C other than SNMP responses to traverse the NAT.
I'm bonding eth0 and eth1 because I require hardware redundancy on VLAN 1. It's not really germane to the conversation about netfilter, policy routing, or SNMP though, so let's not worry about that (unless you feel it necessary).
As you correctly conclude, my chassis includes not just one but two independent internal ethernet switches, switch0 and switch1; each switch has two VLAN's configured: VLAN #1 and VLAN #2. In each switch VLAN1 is connected to and routed "to the rest of the world" via RTR (which in reality is an HSRP pair); VLAN2 is not connected beyond the chassis and is not routed. Each blade in the chassis has a single ethernet link to each internal switch (eth0 to switch0, eth1 to switch1). And the switches are not really filtering anything (other than malformed frames).
>>> So if you strip out all the nonsense and just put the addresses on bond0.1 and bond0.2, then only generate traffic to 192.168.168.anything addresses when you want to use bond0.2 then everything is finished.<<<
The SNMP agents I'm polling are all in 10.0.0.0/8. I'm not seeing how I can avoid a second routing table, using ip rule, iptables, nft, while steering just the outbound SNMP polling traffic to the rest of the world to the NAT blade. It's seems as clear a case for "policy-based routing" as one can draw: "if locally-generated traffic is non-SNMP, use RTR for the outbound next-hop; if locally-generated traffic is SNMP, use nat1 for the outbound next-hop".
If I only configure the IP addresses as you suggest in hostA,B,C... with no mention of nat1's IP address, the kernel on the host will never select bond0.2 for outbound polls to 10.0.0.0/8.
----------8<-------------------
hostA-C and nat1 are all contained within a single chassis; RTR is not within the chassis.
+-------------- INTERNAL network, VLAN 2, 192.168.168.0/24, bond0.2 on all hosts. This VLAN is internal to the chassis and is not routable outside the chassis.
|
|
| +----- EXTERNAL network, VLAN 1, 10.0.0.0/8 network, bond0.1 on all hosts. This VLAN has external connectivity via RTR
| |
| |
| |
V V
~ ~
| +-------+ |
| | | |
+--| hostA |--+
| | | |
| +-------+ |
| |
| +-------+ |
| | | |
+--| hostB |--+ Notes:
| | | |
| +-------+ | hostA-C are not forwarding/routing.
| | other network interfaces omitted for clarity (SAN, DR, etc.).
| +-------+ |
| | | |
+--| hostC |--+
| | | |
| +-------+ |
| | ~
| . | +------+ | (((((((((((((())))))))))))))
| . | | | | ( )
| . +---| RTR |--+--( the rest of the network )
| | | | | ( )
| +-------+ | +------+ | (((((((((((((())))))))))))))
| | | | ~
+--| nat1 |--+
| | | |
| +-------+ |
~ ~
��칻 �&�~�&���+-��ݶ��w��˛���m�������^n�r���z���h����&�� �G���h�(�階�ݢj"���m�����z�ޖ���f���h���~�m�
��.n��������+%������w��{.n����z���)�� jg��� �����ݢj����G���� ���j:+v���w�m������w�������h�����٥
