Hi :) I have a couple of questions regarding nftables rules in combination with a bridge. The questions might be simple but I include the surrounding details to give background. This is my setup: Internet <-----> Gateway router <-----> (eth0) - linux device w/ bridge - (eth1) <-----> LAN router <-----> LAN hosts Addresses: Gateway router: 192.168.10.1/24, e0:b9:e5:da:7b:1c eth0: ip arbitrary but let's assume 192.168.10.203/24, e4:5f:01:a9:57:25 eth1: ip arbitrary, 7c:c2:c6:35:82:08 Lan router: 192.168.10.132/24, 54:af:97:87:eb:b9 br0 (bridge): 0.0.0.0, 7c:c2:c6:35:82:08 I will come back to why I think ip can be arbitrary later. There are 3 things I want for my setup. 1) I am trying to setup a bridge that blindly forwards packets between two interfaces eth0 and eth1 using brctl. This works. I set up my bridge using: ifconfig eth0 promisc up ifconfig eth1 promisc up brctl addbr br0 brctl stp br0 off brctl addif br0 eth0 brctl addif br0 eth1 ifconfig br0 0.0.0.0 echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2) I want the Linux device to be able to communicate with some specific hosts in the internet while running the bridge. As the bridge forwards all ARP broadcast packets, the arp table won't be filled automatically. I manually have to enter the arp entry for the gateway. Then communicating to internet works too. arp -s 192.168.10.1 e0:b9:e5:da:7b:1c -i eth0 arp -s 192.168.10.203 54:af:97:87:eb:b9 -i eth1 3) I want the device to be transparent to the rest of the network. I have a plan for the third step that theoretically works, but it is still giving me trouble. My idea is that when the linux device will send out a packet (some sort of request) of its own to the internet, I want it to rewrite the packet’s source adress to the LAN router’s adress. Now it looks like the packet comes from the LAN router. When the reply comes back from internet, it will be forwarded by the gateway router to the LAN router over the bridge. I want my device to recognise this response packet belonging to itself and ’steal’ it by modifying the packet’s destination adress to the device itself as it’s received, thus internally redirecting the packet to any applications waiting for it instead of forwarding it over the bridge. This way, my device will never be seen by the rest of the network. I can then pick an arbitrary IP adress for my device's ethernet interfaces, and my hope is that this (along with manual ARP table setup) will trick the device into functioning normally, thinking it’s fully visible to the rest of the subnet while it's not. Let’s say I want my device to ping google.com. It pings google’s IP adress, changes source adress on the way out. When the reply comes back, we detect that it has google’s IP adress as source, meaning this is the response we are waiting for, then rewrite the packet’s destination to our own, and hopefully the ping application will pick the response up. Ignore the fact that the LAN router or anything behind it cannot reach google.com anymore since the device will ’steal’ all those packets. Following, I will assume communicating with google.com is the goal. google.com ip address is assumed to be 216.58.207.206. Basically I want a way to modify source/destination adresses on packets using nftables to achieve this. Routing decisions in the bridge layer (link layer) is based on link layer MAC adresses. Meaning if I want to redirect packets from going over the bridge to instead leave the bridge, I'm thinking I need to change the mac adress for packets that I want to route off the bridge to the network layer. So it’s not always enough to just modify the IP adress. For changing the source address of outgoing packets I’ve tried explicitly setting it with ’nft add rule ip filter postrouting ip saddr set 192.168.10.132 ether saddr set 54:af:97:87:eb:b9’ Capturing the packet in wireshark shows the ip address has been altered, but not the mac adress. The mac adress still is eth0 (..57:25). I though about using ’snat’ and ’dnat’ to alter sources/destinations. This however would not suffice because firstly, 'snat' does not modify mac adresses on packets it seems like and secondly, nat:ing is only provided above the bridge layer (according to the table 'bridge family' in https://wiki.nftables.org/wiki-nftables/index.php/Netfilter_hooks), meaning it cannot be used on incoming packets that will traverse the bridge. Perhaps it's not possible to alter link layer addresses on network layer filters. If the outgoing packets from the network layer eventually reach the bridge layer, then we could perhaps modify mac addresses there instead. But looking at the netfilter packet flow diagram (https://commons.wikimedia.org/wiki/File:Netfilter-packet-flow.svg) we see that the outgoing packet does not reach the bridge layer. This was confirmed by me tracing outgoing packets. This raises my first question: Is it even possible to change the mac adress of an outgoing packet with nftables? If so, how? For changing the destination address of incoming packets I’ve tried similar things: 'nft add rule bridge filter prerouting ip daddr set 192.168.10.203 ether daddr set 7c:c2:c6:35:82:08' When tracing incoming ping reply packets from google I now see that both destination addresses have changed (so far it works!). Packets now get routed to the input filter chain in the bridge layer, exactly as I expect. What’s interesting though is that the packet is nowhere to be found after leaving the input chain in the bridge layer. It does not show up at any of the ip layer chains (for example the raw prerouting chain). The packet is completely lost it seems like. Looking at the same packet flow diagram (https://commons.wikimedia.org/wiki/File:Netfilter-packet-flow.svg) we see that after leaving the bridge layer input filter chain, the packet should get routed to one of three possible chains. I've checked all these chains and they all state that 0 packets have gone through them. This leads to my next question. The reason I'm using br0 mac address (..82:08) as new destination is because this is how it was when communicating worked, before trying step 3. For the record I have also tried setting eth0 mac address as new destination but same problem. Is there any way a packet can get dropped in a routing decision in netfilter, or why else is my packet just lost after being accepted in the bridge layer input filter? The following displays how the packet reaches the bridge input filter but not the ip input or prerouting filters. Nftables ruleset (with my own comments): table bridge filter { chain prerouting { # Incoming packet, alter its adresses type filter hook prerouting priority filter; policy drop; ip saddr 216.58.207.206 counter packets 1 bytes 168 # Displays that 1 packet comes from google.com ip saddr 216.58.207.206 ip daddr set 192.168.10.203 ether daddr set 7c:c2:c6:35:82:08 accept # Modify its destination to self } chain input { type filter hook input priority filter; policy accept; ip saddr 216.58.207.206 ip daddr 192.168.10.203 ether daddr 7c:c2:c6:35:82:08 counter packets 1 bytes 168 # Shows that the google packet reached the input chain with altered addresses! } } table ip filter { chain prerouting { type filter hook prerouting priority filter; policy accept; counter packets 0 bytes 0 } chain input { type filter hook input priority filter; policy accept; counter packets 0 bytes 0 } # Above shows that the google response packet neither reached the ip prerouting nor input chain at all! chain postrouting { # For altering the packet's source address on the way out type filter hook postrouting priority filter; policy accept; ip saddr set 192.168.10.132 ether saddr set 54:af:97:87:eb:b9 # As said before, ip address will be changed here but not mac address for some reason (perhaps just not provided in ip layer) } } I've been having problems with this for a week soon and I cannot figure it out. Some hints I’ve found when searching around is: - "The 'filter' table is used for packet filtering, it should never alter packets." (https://unix.stackexchange.com/questions/122847/does-an-in-kernel-firewall-sit-between-the-network-access-layer-and-internet-lay). This sounds reasonable but considering the only chain type of the bridge family is the "filter" type, then this would mean changing mac addresses with nftables on bridge layer is not possible. - The sample code under ’Stateless NAT’ in the following link shows roughly the idea of explicitly setting header values as I tried. They state "You have to disable Connection tracking for modified packets" by specifying ’notrack’ flag at the end of a rule. I tried this as well, making sure the rule has a high priority as recommended in the documentation. No luck, not even any change in behaviour. (https://wiki.nftables.org/wiki-nftables/index.php/Performing_Network_Address_Translation_(NAT)#Stateful_NAT) - I'm seeing that my br0 bridge automatically receives the same mac address as eth1. At first I thought this might be a problem but at the same time communicating works fine when the bridge is running (pinging google.com works fine), just not when the nftables rules trying to alter header values (step 3). So there shouldn't be a problem with routing itself. - Perhaps it is possible to redirect an incoming packet from the bridge layer to the ip layer in another way than my attempt of modifying mac address. I have not found any other way though. To reiterate my questions: 1) Is it possible to change the mac adress of an outgoing packet with nftables? If so, how? 2) Is there any way a packet can get dropped in a routing decision in netfilter, or why else is my packet just lost after going through the bridge layer input filter? For extra clarity: Along the way I've been looking at these flow charts and information when explaining where packets get lost (both already referenced above): https://commons.wikimedia.org/wiki/File:Netfilter-packet-flow.svg https://wiki.nftables.org/wiki-nftables/index.php/Netfilter_hooks Thanks a lot for any guidance!
