Hello,
I am trying to implement QoS on a physical interface (e.g. eth0) that
carries multiple VLANs (e.g. there are multiple eth0.VID). I only want
to setup QoS (via tc) on the physical interface and not on the VLAN
interfaces (if that is possible - this is basically my key question).
Traffic shall be filtered by L3 and L4 information.
My main problem is: I am trying to set up filters (flower or u32) on a
physical interface only, which shall match both tagged and untagged
traffic passing through that physical interface. This does not work,
i.e. I am not getting any filter matches for VLAN-tagged traffic on
the filters attached to the physical interface. I am not sure whether
it is a misunderstanding or misconfiguration on my side or maybe even
an (intended) design limitation or feature.
In the following I will illustrate a minimal setup to describe my
problem, followed by my observations, and question at the end.
# Setup (not the actual one, but I recon this should be enough for
illustration)
I am implementing static priority transmission selection via tc-prio
with 8 bands. Traffic shall be mapped to bands based on L3 and L4
information (IPv6 addresses and UDP ports). In all (minimal) scenarios
below a tc-flower filter is used which classifies matching traffic to
prio qdisc queue 1:4. With tc-u32 I am seeing similar results.
I am running Archlinux with a 4.17.11 Kernel and iproute2 version
4.17.0-1.
## Terminology
I hope this will help with the description of my problem:
- physical interface: The "normal" (untagged) network interface, e.g.
eth0
- VLAN interface: A VLAN interface on top of a physical interface,
e.g. eth0.VID
## Minimal example
The following script serves as a minimal example to illustrate my
problem. For the three scenarios below, IF, IP_SRC, and IP_DST are
adjusted.
~~~~
#!/usr/bin/env bash
IF="enp0s31f6"
IP_SRC="fd00:0:1:41::11"
IP_DST="fd00:0:1:41::10"
echo "Using $IF"
# Add prio qdisc and make sure that we do not have any other qdiscs on
any VLAN interface (quick and dirty)
tc qdisc del dev enp0s31f6 root
tc qdisc del dev enp0s31f6.65 root
tc qdisc add dev $IF root handle 1: prio bands 8 priomap 7 6 5 4 3 2 1
0 7 7 7 7 7 7 7 7
# Add filter
tc filter add dev $IF protocol ipv6 parent 1:0 prio 1 flower ip_proto
udp src_ip ${IP_SRC}/128 dst_ip ${IP_DST}/128 src_port 50000 dst_port
60000 classid 1:4 action pass
# Sent traffic via (OpenBSD) netcat (NOTE: keep this low enough to
avoid fragmentation)
dd if=/dev/zero bs=100 count=1 | nc -u -p 50000 ${IP_DST} 60000
# Show interface config
ip a s
# Print tc statistics
tc -s qdisc ls dev $IF
tc -s class ls dev $IF
tc -s filter ls dev $IF
~~~~
# Scenario 1: QoS on VLAN interface with VLAN-tagged traffic
- IF="enp0s31f6.65"
- IP_SRC="fd00:0:0:41::11"
- IP_DST="fd00:0:0:41::10"
- tc filter works as expected (one packet seen by the filter)
## Console output
~~~~
# ./tc_test_minimal_flower.sh
Using enp0s31f6.65
Error: Cannot delete qdisc with handle of zero.
1+0 records in
1+0 records out
100 bytes copied, 3.7431e-05 s, 2.7 MB/s
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp0s31f6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
fq_codel state UP group default qlen 1000
link/ether 50:7b:9d:ab:12:96 brd ff:ff:ff:ff:ff:ff
inet 192.168.165.11/24 scope global enp0s31f6
valid_lft forever preferred_lft forever
inet6 fd00:0:1:41::11/64 scope global
valid_lft forever preferred_lft forever
inet6 fe80::527b:9dff:feab:1296/64 scope link
valid_lft forever preferred_lft forever
3: wlp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state
UP group default qlen 1000
link/ether a4:34:d9:94:24:4c brd ff:ff:ff:ff:ff:ff
inet 192.168.32.238/24 brd 192.168.32.255 scope global
noprefixroute wlp4s0
valid_lft forever preferred_lft forever
inet6 fe80::a634:d9ff:fe94:244c/64 scope link
valid_lft forever preferred_lft forever
4: wwp0s20f0u2i12: <BROADCAST,MULTICAST,NOARP> mtu 1500 qdisc noop
state DOWN group default qlen 1000
link/ether 8e:67:2d:3c:14:46 brd ff:ff:ff:ff:ff:ff
5: enp0s31f6.65@enp0s31f6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500
qdisc prio state UP group default qlen 1000
link/ether 50:7b:9d:ab:12:96 brd ff:ff:ff:ff:ff:ff
inet 192.168.65.11/24 scope global enp0s31f6.65
valid_lft forever preferred_lft forever
inet6 fd00:0:0:41::11/64 scope global
valid_lft forever preferred_lft forever
inet6 fe80::527b:9dff:feab:1296/64 scope link
valid_lft forever preferred_lft forever
qdisc prio 1: root refcnt 2 bands 8 priomap 7 6 5 4 3 2 1 0 7 7 7 7 7
7 7 7
Sent 162 bytes 1 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
class prio 1:1 parent 1:
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
class prio 1:2 parent 1:
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
class prio 1:3 parent 1:
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
class prio 1:4 parent 1:
Sent 162 bytes 1 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
class prio 1:5 parent 1:
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
class prio 1:6 parent 1:
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
class prio 1:7 parent 1:
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
class prio 1:8 parent 1:
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
filter parent 1: protocol ipv6 pref 1 flower chain 0
filter parent 1: protocol ipv6 pref 1 flower chain 0 handle 0x1
classid 1:4
eth_type ipv6
ip_proto udp
dst_ip fd00:0:0:41::10
src_ip fd00:0:0:41::11
dst_port 60000
src_port 50000
not_in_hw
action order 1: gact action pass
random type none pass val 0
index 1 ref 1 bind 1 installed 0 sec used 0 sec
Action statistics:
Sent 162 bytes 1 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
~~~~
## tcpdump output (tcpdump -ni enp0s31f6 -xx)
~~~~
10:04:21.055112 IP6 fd00:0:0:41::11.50000 > fd00:0:0:41::10.60000:
UDP, length 100
0x0000: 000e c6da b35d 507b 9dab 1296 8100 0041
0x0010: 86dd 6000 d6ab 006c 1140 fd00 0000 0000
0x0020: 0041 0000 0000 0000 0011 fd00 0000 0000
0x0030: 0041 0000 0000 0000 0010 c350 ea60 006c
0x0040: fb21 0000 0000 0000 0000 0000 0000 0000
0x0050: 0000 0000 0000 0000 0000 0000 0000 0000
0x0060: 0000 0000 0000 0000 0000 0000 0000 0000
0x0070: 0000 0000 0000 0000 0000 0000 0000 0000
0x0080: 0000 0000 0000 0000 0000 0000 0000 0000
0x0090: 0000 0000 0000 0000 0000 0000 0000 0000
0x00a0: 0000 0000 0000
~~~~
# Scenario 2: QoS on physical interface with VLAN-tagged traffic
- IF="enp0s31f6"
- IP_SRC="fd00:0:0:41::11"
- IP_DST="fd00:0:0:41::10"
- tc filter does not work (no packets seen by the filter)
## Console output
~~~~
# ./tc_test_minimal_flower.sh
Using enp0s31f6
Error: Cannot delete qdisc with handle of zero.
1+0 records in
1+0 records out
100 bytes copied, 3.6501e-05 s, 2.7 MB/s
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp0s31f6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc prio
state UP group default qlen 1000
link/ether 50:7b:9d:ab:12:96 brd ff:ff:ff:ff:ff:ff
inet 192.168.165.11/24 scope global enp0s31f6
valid_lft forever preferred_lft forever
inet6 fd00:0:1:41::11/64 scope global
valid_lft forever preferred_lft forever
inet6 fe80::527b:9dff:feab:1296/64 scope link
valid_lft forever preferred_lft forever
3: wlp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state
UP group default qlen 1000
link/ether a4:34:d9:94:24:4c brd ff:ff:ff:ff:ff:ff
inet 192.168.32.238/24 brd 192.168.32.255 scope global
noprefixroute wlp4s0
valid_lft forever preferred_lft forever
inet6 fe80::a634:d9ff:fe94:244c/64 scope link
valid_lft forever preferred_lft forever
4: wwp0s20f0u2i12: <BROADCAST,MULTICAST,NOARP> mtu 1500 qdisc noop
state DOWN group default qlen 1000
link/ether 8e:67:2d:3c:14:46 brd ff:ff:ff:ff:ff:ff
5: enp0s31f6.65@enp0s31f6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500
qdisc noqueue state UP group default qlen 1000
link/ether 50:7b:9d:ab:12:96 brd ff:ff:ff:ff:ff:ff
inet 192.168.65.11/24 scope global enp0s31f6.65
valid_lft forever preferred_lft forever
inet6 fd00:0:0:41::11/64 scope global
valid_lft forever preferred_lft forever
inet6 fe80::527b:9dff:feab:1296/64 scope link
valid_lft forever preferred_lft forever
qdisc prio 1: root refcnt 2 bands 8 priomap 7 6 5 4 3 2 1 0 7 7 7 7 7
7 7 7
Sent 162 bytes 1 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
class prio 1:1 parent 1:
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
class prio 1:2 parent 1:
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
class prio 1:3 parent 1:
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
class prio 1:4 parent 1:
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
class prio 1:5 parent 1:
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
class prio 1:6 parent 1:
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
class prio 1:7 parent 1:
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
class prio 1:8 parent 1:
Sent 162 bytes 1 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
filter parent 1: protocol ipv6 pref 1 flower chain 0
filter parent 1: protocol ipv6 pref 1 flower chain 0 handle 0x1
classid 1:4
eth_type ipv6
ip_proto udp
dst_ip fd00:0:0:41::10
src_ip fd00:0:0:41::11
dst_port 60000
src_port 50000
not_in_hw
action order 1: gact action pass
random type none pass val 0
index 2 ref 1 bind 1 installed 0 sec used 0 sec
Action statistics:
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
~~~~
## tcpdump output (tcpdump -ni enp0s31f6 -xx)
~~~~
10:14:22.564206 IP6 fd00:0:0:41::11.50000 > fd00:0:0:41::10.60000:
UDP, length 100
0x0000: 000e c6da b35d 507b 9dab 1296 8100 0041
0x0010: 86dd 6004 8fc9 006c 1140 fd00 0000 0000
0x0020: 0041 0000 0000 0000 0011 fd00 0000 0000
0x0030: 0041 0000 0000 0000 0010 c350 ea60 006c
0x0040: fb21 0000 0000 0000 0000 0000 0000 0000
0x0050: 0000 0000 0000 0000 0000 0000 0000 0000
0x0060: 0000 0000 0000 0000 0000 0000 0000 0000
0x0070: 0000 0000 0000 0000 0000 0000 0000 0000
0x0080: 0000 0000 0000 0000 0000 0000 0000 0000
0x0090: 0000 0000 0000 0000 0000 0000 0000 0000
0x00a0: 0000 0000 0000
~~~~
# Scenario 3: QoS on physical interface with untagged traffic
- IF="enp0s31f6"
- IP_SRC="fd00:0:1:41::11"
- IP_DST="fd00:0:1:41::10"
- tc filter works as expected (one packet seen by the filter)
## Console output
~~~~
# ./tc_test_minimal_flower.sh
Using enp0s31f6
Error: Cannot delete qdisc with handle of zero.
1+0 records in
1+0 records out
100 bytes copied, 2.6061e-05 s, 3.8 MB/s
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp0s31f6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc prio
state UP group default qlen 1000
link/ether 50:7b:9d:ab:12:96 brd ff:ff:ff:ff:ff:ff
inet 192.168.165.11/24 scope global enp0s31f6
valid_lft forever preferred_lft forever
inet6 fd00:0:1:41::11/64 scope global
valid_lft forever preferred_lft forever
inet6 fe80::527b:9dff:feab:1296/64 scope link
valid_lft forever preferred_lft forever
3: wlp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state
UP group default qlen 1000
link/ether a4:34:d9:94:24:4c brd ff:ff:ff:ff:ff:ff
inet 192.168.32.238/24 brd 192.168.32.255 scope global
noprefixroute wlp4s0
valid_lft forever preferred_lft forever
inet6 fe80::a634:d9ff:fe94:244c/64 scope link
valid_lft forever preferred_lft forever
4: wwp0s20f0u2i12: <BROADCAST,MULTICAST,NOARP> mtu 1500 qdisc noop
state DOWN group default qlen 1000
link/ether 8e:67:2d:3c:14:46 brd ff:ff:ff:ff:ff:ff
5: enp0s31f6.65@enp0s31f6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500
qdisc noqueue state UP group default qlen 1000
link/ether 50:7b:9d:ab:12:96 brd ff:ff:ff:ff:ff:ff
inet 192.168.65.11/24 scope global enp0s31f6.65
valid_lft forever preferred_lft forever
inet6 fd00:0:0:41::11/64 scope global
valid_lft forever preferred_lft forever
inet6 fe80::527b:9dff:feab:1296/64 scope link
valid_lft forever preferred_lft forever
qdisc prio 1: root refcnt 2 bands 8 priomap 7 6 5 4 3 2 1 0 7 7 7 7 7
7 7 7
Sent 162 bytes 1 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
class prio 1:1 parent 1:
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
class prio 1:2 parent 1:
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
class prio 1:3 parent 1:
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
class prio 1:4 parent 1:
Sent 162 bytes 1 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
class prio 1:5 parent 1:
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
class prio 1:6 parent 1:
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
class prio 1:7 parent 1:
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
class prio 1:8 parent 1:
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
filter parent 1: protocol ipv6 pref 1 flower chain 0
filter parent 1: protocol ipv6 pref 1 flower chain 0 handle 0x1
classid 1:4
eth_type ipv6
ip_proto udp
dst_ip fd00:0:1:41::10
src_ip fd00:0:1:41::11
dst_port 60000
src_port 50000
not_in_hw
action order 1: gact action pass
random type none pass val 0
index 2 ref 1 bind 1 installed 0 sec
Action statistics:
Sent 162 bytes 1 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
~~~~
## tcpdump output (tcpdump -ni enp0s31f6 -xx)
~~~~
10:17:34.262644 IP6 fd00:0:1:41::11.50000 > fd00:0:1:41::10.60000:
UDP, length 100
0x0000: 000e c6da b35d 507b 9dab 1296 86dd 6005
0x0010: 0645 006c 1140 fd00 0000 0001 0041 0000
0x0020: 0000 0000 0011 fd00 0000 0001 0041 0000
0x0030: 0000 0000 0010 c350 ea60 006c fb23 0000
0x0040: 0000 0000 0000 0000 0000 0000 0000 0000
0x0050: 0000 0000 0000 0000 0000 0000 0000 0000
0x0060: 0000 0000 0000 0000 0000 0000 0000 0000
0x0070: 0000 0000 0000 0000 0000 0000 0000 0000
0x0080: 0000 0000 0000 0000 0000 0000 0000 0000
0x0090: 0000 0000 0000 0000 0000 0000 0000 0000
0x00a0: 0000
~~~~
# Observations
- Applying filters that parse a packet's payload (I tried u32 and
flower so far) to the physical interface does not seem to work on
VLAN-tagged traffic passing the physical interface.
- Applying the same filters to the VLAN interface works as expected.
- Or in other words: if the filtering uses the IP subnet assigned to
the interfaces, to which the filter is attached, it works. Maybe this
is even the intended behavior for tc filters (I am new to this and
want to understand it)?
- Note that I do know that what I am trying to do could also be
achieved by marking traffic in netfilter and tc-fw. However, we are
currently in an evaluation phase and would like to investigate a
tc-only option as well.
# Questions
1. Is my understanding of
[http://lartc.vger.kernel.narkive.com/I6VpagXF/vlan-interfaces-and-tc]
correct, i.e. that when QoS is applied to the physical interface it
applies to all traffic passing this interface, and when it is applied
to the VLAN inferace it only applies to traffic on that particular
VLAN interface and at the physical interface (if no QoS is added
there) traffic form all potential VLAN interfaces passes the physical
interface "uncontrolled"?
2. Is there a way around this issue (observations above) without
resorting to netfilter (fwmarks) and tc-fw, i.e. to define QoS only on
the physical interface and use u32 or flower to classify untagged
traffic from the physical interface as well as VLAN-tagged traffic
from VLAN interfaces?
Maybe an alternative would be to filter on L3 and L4 payload (u32 of
flower) at the VLAN interface and use tc actions to additionally
mark/edit packets. Then, at the physical interface, matching on the
marked or edited properties could be performed (e.g. set SKB priority
at the VLAN interface and then use either tc-prio's built-in filtering
or basic match "meta(priority eq 0xXX)" at the physical interface).
Best regards,
Daniel
--
To unsubscribe from this list: send the line "unsubscribe lartc" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
