Hello, > Ouch. That sounds serious. - On my side of the pond, cease and > desist letters usually are a friendly "stop, or else" type. In fact, > most ISPs over here need to send three before they can actually > terminate your connection. (RIAA and MPAA are notorious for causing > ISPs to send such letters.) Yeah, they'd never shut you down here. Lawyers will just send you one expensive letter after another ;). But that's another topic. Things are almost resolved, just some trials needed to proove the new laws. > > The thing that is important, at least for me to understand, is if the > GRE tunnels are between the CPE (OpenWRT routers) and your VMs are > across the internet. Verses, what I thought when this thread started, > your border routers across your internal LAN to your VMs on your own > hosts. > > The pertinent part is if the GRE is crossing the internet vs your own > LAN. > The GRE-Tunnels are always online between VM on the internet. Never in my private LAN. > >> Typical situation: >> >> VM (pubilc IP) <---> Hypervisor <---the internet---> Hypervisor <---> >> VM (public OR private IP) > > Please confirm the GRE tunnels are between the VM on the left (with a > public IP) and the VM on the right (with a public OR private IP). Exactly, from the very left to the very right. From one VM to another. Some hosts aren't virtualized, but that doesn't make a difference. > > I am somewhat questioning the need for private IPs vs public IPs on > the VMs. (I'm still trying to wrap my head around things.) It is not needed. We could book another public IP, assign it to the VM and request a tunneld endpoint change to the new IP. But I'd like to understand, how to diagnose those kinds of problems and what worked once flawlessly, should still work. :) > > That's odd. > > NAT shouldn't eat anything. > > NAT should alter IP addresses or not alter them. But NAT itself > should not cause traffic to disappear. That sounds like something > else. I just don't know what. Jep, completly wired. > > > Does dmesg give any output that might shed some light on things? Nothing in dmesg, nothing in syslog. > > What if you change your tcpdump filter to just look for GRE (protocol > 47) traffic? It will likely match more than what is necessary. But > hopefully it will show that packets are being NATed in an unexpected > way. In a way that doesn't function on the other end. There is nothing there. I ran the ping at hundret packages per second to have enough packets to find between the hundrets of packets going through here. There were just the two unnatted ICMP request packages, we've seen before, followed by the next two with the next sequential number. Nothing inbetween. Nothing that could be a wrongly natted or broken package. > > I'm thinking that it might not block traffic that you want filtered. > Though I'm assuming that you do want to filter / block some traffic. > (All of the firewalls that I've written were to filter all but the > desired traffic.) No, we have very strict network neutrality. We filter absolutely nothing, no traffic shaping, no blocked ports. > > That sounds like standard DNAT to me. Yes. > > Along with SNAT or MASQUERADE going the other direction. Yes. > > There's a way to get connection tracking which is used by NATing > information out of the kernel. > > Look into conntrackd and the conntrack command. You can use the > command to get information about what the connection tracking > subsystem is doing. I tried this one. The broken tunnels are marked with „UNREPLIED“. Well, that sounds reasonable, as there's nothing coming back. root@unimatrixzero ~ # conntrack -L|grep gre conntrack v1.4.3 (conntrack-tools): 97 flow entries have been shown. 3:gre 47 176 src=185.66.193.1 dst=176.9.38.158 srckey=0x0 dstkey=0x0 src=176.9.38.158 dst=185.66.193.1 srckey=0x0 dstkey=0x0 [ASSURED] mark=0 use=1 9:gre 47 29 src=185.66.194.1 dst=176.9.38.150 srckey=0x0 dstkey=0x0 [UNREPLIED] src=176.9.38.150 dst=185.66.194.1 srckey=0x0 dstkey=0x0 mark=0 use=1 14:gre 47 29 src=185.66.194.0 dst=176.9.38.150 srckey=0x0 dstkey=0x0 [UNREPLIED] src=176.9.38.150 dst=185.66.194.0 srckey=0x0 dstkey=0x0 mark=0 use=1 29:gre 47 179 src=46.4.80.131 dst=176.9.38.158 srckey=0x0 dstkey=0x0 src=176.9.38.158 dst=46.4.80.131 srckey=0x0 dstkey=0x0 [ASSURED] mark=0 use=1 30:gre 47 177 src=185.66.193.0 dst=176.9.38.158 srckey=0x0 dstkey=0x0 src=176.9.38.158 dst=185.66.193.0 srckey=0x0 dstkey=0x0 [ASSURED] mark=0 use=1 40:gre 47 29 src=185.66.195.0 dst=176.9.38.150 srckey=0x0 dstkey=0x0 [UNREPLIED] src=176.9.38.150 dst=185.66.195.0 srckey=0x0 dstkey=0x0 mark=0 use=1 60:gre 47 26 src=88.198.51.94 dst=176.9.38.150 srckey=0x0 dstkey=0x0 [UNREPLIED] src=176.9.38.150 dst=88.198.51.94 srckey=0x0 dstkey=0x0 mark=0 use=1 62:gre 47 179 src=185.66.194.0 dst=176.9.38.158 srckey=0x0 dstkey=0x0 src=176.9.38.158 dst=185.66.194.0 srckey=0x0 dstkey=0x0 [ASSURED] mark=0 use=1 69:gre 47 177 src=185.66.193.1 dst=176.9.38.150 srckey=0x0 dstkey=0x0 src=192.168.10.62 dst=185.66.193.1 srckey=0x0 dstkey=0x0 [ASSURED] mark=0 use=1 74:gre 47 174 src=192.168.10.62 dst=185.66.193.0 srckey=0x0 dstkey=0x0 src=185.66.193.0 dst=176.9.38.150 srckey=0x0 dstkey=0x0 [ASSURED] mark=0 use=1 75:gre 47 169 src=185.66.194.1 dst=176.9.38.158 srckey=0x0 dstkey=0x0 src=176.9.38.158 dst=185.66.194.1 srckey=0x0 dstkey=0x0 [ASSURED] mark=0 use=1 80:gre 47 179 src=176.9.38.158 dst=176.9.38.156 srckey=0x0 dstkey=0x0 src=176.9.38.156 dst=176.9.38.158 srckey=0x0 dstkey=0x0 [ASSURED] mark=0 use=1 82:gre 47 179 src=185.66.195.1 dst=176.9.38.158 srckey=0x0 dstkey=0x0 src=176.9.38.158 dst=185.66.195.1 srckey=0x0 dstkey=0x0 [ASSURED] mark=0 use=1 85:gre 47 179 src=46.4.80.131 dst=176.9.38.156 srckey=0x0 dstkey=0x0 src=176.9.38.156 dst=46.4.80.131 srckey=0x0 dstkey=0x0 [ASSURED] mark=0 use=1 91:gre 47 179 src=185.66.195.0 dst=176.9.38.158 srckey=0x0 dstkey=0x0 src=176.9.38.158 dst=185.66.195.0 srckey=0x0 dstkey=0x0 [ASSURED] mark=0 use=1 95:gre 47 177 src=192.168.10.62 dst=185.66.195.1 srckey=0x0 dstkey=0x0 src=185.66.195.1 dst=176.9.38.150 srckey=0x0 dstkey=0x0 [ASSURED] mark=0 use=1 Do you have any conntrack tricks to look into this further? Maybe we should start looking into the code, the package goes through. Are you familiar with that part of the kernel? So far I only found that one function, I copied a few days earlier. Maybe the „standard“ nat code fails here, because GRE is less than an UDP package. Or because it's stateless. Just a wild guess. Maybe it has something to do with weather the first package in this tunnel is incoming or outgoing. This is random, because you never know, which of the two BGP daemons try to get a connection first. Bye, Matthias -- To unsubscribe from this list: send the line "unsubscribe lartc" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
