I am experimenting with using ipvtap devices for virtual machines &
user-space VPN. I want them to be able to configure their own IP/IPv6
addresses from the network using DHCP, DHCPv6 and SLAAC. IPVlan-based
devices can only receive unicast traffic on the address(es) that are
associated with the interface in the kernel.
I figured I could enable this with a program that snooped DHCP/ICMPv6
packets and configured the addresses it observed interfaces negotiating. I
thought that netfilter queues[1] would be a good way to intercept the
packets I was interested in. However I'm having a bear of a time getting
the client packets (DHCPDISCOVER, nd-router-solicit, etc) that the
VMs write to the tap device.
Here's the the part of my ruleset for SLAAC (full set here[2]):
table inet autoconf {
chain output {
type filter hook output priority 0; policy accept;
counter meta protocol ip6 icmpv6 type { nd-router-solicit } queue;
}
chain input {
type filter hook input priority 0; policy accept;
counter meta protocol ip6 icmpv6 type { nd-router-advert } queue;
}
}
And I have a user-space program listening on queue 0 that will configure
link-local addresses on the outdev observed in solicit messages, and
mask those with any prefixes learned in router-advertisements. My program
receives the router adverts (with the physical interface as iif), but not
the solicits.
I've done a lot of tweaking and experimentation, enabling tracing and
such. I can see the packets passing through the "ingress" and "egress"
hooks of the "netdev" family[3]. Here are two example traces of packets I
could use to learn the link-local and global address(es) that the client
is likely to use:
trace id 99f29220 netdev trace egress packet: oif "ipvtap22476" @nh,0,320 0x6000000000203afffe800000000000005e5f67fffefeae20ff020000000000000000000000000001 @th,0,160 0x8800002320000000fe800000000000005e5f67ff
trace id 8a307430 netdev trace ingress packet: iif "ipvtap22476" ether saddr 88:0f:a2:2f:09:60 ether daddr 33:33:00:00:00:01 ip6 saddr fe80::8a0f:a2ff:fe2f:960 ip6 daddr ff02::1 ip6 dscp cs0 ip6 ecn not-ect ip6 hoplimit 255 ip6 flowlabel 0 ip6 nexthdr ipv6-icmp ip6 length 136 icmpv6 type nd-router-advert icmpv6 code no-route icmpv6 parameter-problem 1078460596 @th,64,96 0x30440c0
However, the netdev family does not seem to support queueing to userspace.
All devices are in the same network namespace, so the packets should all
be passing through the same netfilter stack, but I cannot see the outbound
multicast/broadcast DHCP and ICMPv6 packets in any inet family hooks.
I've been plodding through the ipvlan code in the kernel, but it's
slow going. Can anyone help me understand where these packets enter
the kernel and if they pass through any hooks in the ip/ip6 family? Or
alternatively, another way to observe packets written to a tap device
that doesn't involve opening a raw socket for each device.
David
[1]: https://wiki.nftables.org/wiki-nftables/index.php/Queueing_to_userspace
[2]: https://paste.sr.ht/~droyo/afa648300fdd84e41eb083644501d663b56e3477
[3]: https://paste.sr.ht/~droyo/f01f17481efe4f6326a642c8f97dc19662fcf69c
[4]: https://wiki.nftables.org/wiki-nftables/index.php/Netfilter_hooks
