Re: Traffic shaping at 10~300mbps at a 10Gbps link

Linux Advanced Routing and Traffic Control

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi,

this question has been discussed a bit on a different mailing list[1],
but the information given there[2][3] has not yet reached the LARTC list.
So here are URLs that may or may not provide insight into the problem,
but no solution so far:

[1] https://lists.bufferbloat.net/listinfo/bloat
[2] https://lists.bufferbloat.net/pipermail/bloat/2021-June/016440.html
[3] https://lists.bufferbloat.net/pipermail/bloat/2021-June/016441.html

I myself have no idea regarding the problem, sorry.

Thanks,
Erik
-- 
In the beginning, there was static routing.
                        -- RFC 1118


On Wed, Jun 16, 2021 at 12:43:12PM +0100, cronolog+lartc wrote:
> Hi,
> 
> The initial query mentioned the issue occurs at around 4.5Gbps throughput or higher.
> 
> I wonder if a 32-bit value is overflowing somewhere. If the traffic rate or some other
> value used for internal calculations of the qdisc uses an unsigned 32-bit value, then
> this can only go up to 4294967295 which is just short of 4.3Gbps. So would make sense
> problems start to occur when you go beyond this throughput rate. I don't think the
> filters or quantity of filters would be the problem, they just classify the traffic.
> The problem would likely be within how a qdisc is implemented and how and when it
> chooses to dequeue packets for transmission. More likely the bug is within HTB than
> SFQ, as I don't think SFQ cares about throughput rate, but that's just my guess.
> 
> How are the qdiscs/classes created? It would be interesting to compare with the
> configuration from the person who has it working at 5Gbps.
> 
> I also thought tc/iproute2 supported 64-bit values for traffic rate. The tc man page
> says otherwise i.e. only 32-bit, but the source code clearly shows 64-bit support for
> measuring rate, so I assume the man page is out of date. So if possible, might also be
> worth trying the setup on a newer kernel and version of iproute2 than what's used by
> default in the version of Ubuntu mentioned.
> 
> HFSC is also an option as already suggested. I've not used it myself so not familiar
> with its configuration or how it compares to HTB+SFQ. But if it doesn't have what
> appears to be a 32-bit calculation bug, then that's good.
> 
> Regards,
> 
> Ali
> 
> 
> On 14/06/2021 17:13, Ethy H. Brito wrote:
> 
> >On Sat, 12 Jun 2021 14:24:21 +0300
> >Anatoly Muliarski <x86ever@xxxxxxxxx> wrote:
> >
> >>HTB does not work correctly on high speed links.
> >>Try to use HFSC.
> >I've read some reports about problem of CPU consumption using HFSC+SFQ.
> >
> >What qdisc do you use associated with each leaf (users) HFSC classes?
> >
> >Does HFSC inner classes need associated queue disciplines?
> >
> >Cheers
> >
> >Ethy
> >
> >>2021-06-10 1:13 GMT+03:00, Ethy H. Brito <ethy.brito@xxxxxxxxxxxx>:
> >>>On Wed, 9 Jun 2021 21:57:30 +0200
> >>>Adam Niescierowicz <adam.niescierowicz@xxxxxxxxxx> wrote:
> >>>>W dniu 09.06.2021 o 21:30, Ethy H. Brito pisze:
> >>>>>On Wed, 9 Jun 2021 21:11:03 +0200
> >>>>>Adam Niescierowicz <adam.niescierowicz@xxxxxxxxxx> wrote:
> >>>>>>How you make filters, it's one level or multilevel based?
> >>>>>Multi level.
> >>>>>
> >>>>>First test IPv4, IPv6 or Other Protocols.
> >>>>>Then test for two cidr "/19" and one /20 blocks we use.
> >>>>>Then for every /24 block inside the selected cidr block.
> >>>>>
> >>>>>I can send some fragments of each "test" block, if you want.
> >>>>Please send it will be easier.
> >>>>
> >>>>We have 5+ Gbps at peak, 12core cpu and it's working. We don't use ipv6
> >>>>on this router, but we use PPPoE.
> >>>>
> >>>>
> >>>>What's kernel version?, our is old 4.17.
> >>># lsb_release -a
> >>>No LSB modules are available.
> >>>Distributor ID:	Ubuntu
> >>>Description:	Ubuntu 20.04.2 LTS
> >>>Release:	20.04
> >>>Codename:	focal
> >>>
> >>># tc -V
> >>>tc utility, iproute2-ss200127
> >>>
> >>># uname -a
> >>>Linux quiron 5.4.0-66-generic #74-Ubuntu SMP Wed Jan 27 22:54:38 UTC 2021
> >>>x86_64 x86_64 x86_64 GNU/Linux
> >>>
> >>>
> >>>Here we go.
> >>>
> >>>eth0 is the internal interface (send packets to users - download)
> >>>The rules are inserted in the order bellow. There are no inversions.
> >>>
> >>>Just a very few lines were copied here.
> >>>I can send a WeTransfer link for those who may need the full script file.
> >>>
> >>>  ---------------------------8<------------------------------------
> >>>
> >>>/sbin/tc filter add dev eth0 parent 1:0 prio 1 protocol ipv6 u32
> >>>
> >>>/sbin/tc filter add dev eth0 parent 1:0 prio 2 protocol ip u32
> >>>
> >>># Some service networks
> >>>/sbin/tc filter add dev eth0 protocol ipv4 parent 1: pref 2 \
> >>>         u32 match ip dst 100.64.254.0/24 \
> >>>         flowid 1:fffe
> >>>/sbin/tc filter add dev eth0 protocol ipv6 parent 1: pref 1 \
> >>>         u32 match ip6 dst fe80::/10 \
> >>>         flowid 1:fffe
> >>>
> >>>#
> >>>/sbin/tc filter add dev eth0 parent 1:0 prio 99 handle 1: u32 divisor 4
> >>>
> >>>/sbin/tc filter add dev eth0 parent 1:0 prio 99 handle 10: u32 divisor 256
> >>>
> >>>/sbin/tc filter add dev eth0 parent 1:0 protocol ipv6 prio 1 u32 ht 1: match
> >>>ip6 dst 2001:12c4:a10:e000::/51  hashkey mask 0x0000ff00 at 28  link 10:
> >>>
> >>>/sbin/tc filter add dev eth0 parent 1:0 prio 99 handle 100: u32 divisor 256
> >>>
> >>>/sbin/tc filter add dev eth0 parent 1:0 protocol ipv6 prio 1 u32 ht 10:e0:
> >>>match ip6 dst 2001:12c4:a10:e000::/56 hashkey mask 0x000000ff at 28 link
> >>>100:
> >>>
> >>>... links 101 (2001:12c4:a10:e100::/56), 102 (2001:12c4:a10:e200::/56) ...
> >>>up to
> >>>
> >>>/sbin/tc filter add dev eth0 parent 1:0 protocol ipv6 prio 1 u32 ht 10:ff:
> >>>match ip6 dst 2001:12c4:a10:ff00::/56 hashkey mask 0x000000ff at 28 link
> >>>11f:
> >>>
> >>>/sbin/tc filter add dev eth0 parent 1:0 prio 99 handle 11: u32 divisor 256
> >>>
> >>>/sbin/tc filter add dev eth0 parent 1:0 protocol ipv6 prio 1 u32 ht 1: match
> >>>ip6 dst 2001:12c4:6440::/51  hashkey mask 0x0000ff00 at 28  link 11:
> >>>
> >>>
> >>>/sbin/tc filter add dev eth0 parent 1:0 prio 99 handle 120: u32 divisor 256
> >>>/sbin/tc filter add dev eth0 parent 1:0 protocol ipv6 prio 1 u32 ht 11:00:
> >>>match ip6 dst 2001:12c4:6440:000::/56 hashkey mask 0x000000ff at 28 link
> >>>120:
> >>>
> >>>... same here up to link 13f (2001:12c4:6440:1f00::/56):
> >>>More IPv6 filters like these follows to others CIDRs.
> >>>
> >>>Now for IPv4 /24 blocks.
> >>>
> >>>/sbin/tc filter add dev eth0 parent 1:0 prio 99 handle 2: u32 divisor 4
> >>>
> >>># Hash Table para 256 Blocos /24 de endereços IP para o macro bloco
> >>>10.16.224.0/19
> >>>
> >>>/sbin/tc filter add dev eth0 parent 1:0 prio 99 handle 13: u32 divisor 256
> >>>
> >>>/sbin/tc filter add dev eth0 parent 1:0 protocol ip prio 2 u32 ht 2: match
> >>>ip dst 10.16.224.0/19  hashkey mask 0x0000ff00 at 16  link 13:
> >>>
> >>>/sbin/tc filter add dev eth0 parent 1:0 prio 99 handle 150: u32 divisor 256
> >>>/sbin/tc filter add dev eth0 parent 1:0 protocol ip prio 2 u32 ht 13:e0:
> >>>match ip dst 10.16.224.0/24 hashkey mask 0x000000ff at 16 link 150:
> >>>
> >>>... link 151 (10.16.225.0/24), 153 (10.16.226.0/24) ... up to
> >>>
> >>>/sbin/tc filter add dev eth0 parent 1:0 prio 99 handle 16f: u32 divisor 256
> >>>/sbin/tc filter add dev eth0 parent 1:0 protocol ip prio 2 u32 ht 13:ff:
> >>>match ip dst 10.16.255.0/24 hashkey mask 0x000000ff at 16 link 16f:
> >>>
> >>>Other CIDRs blocks for 100.64.0/19 follows like...
> >>>/sbin/tc filter add dev eth0 parent 1:0 prio 99 handle 14: u32 divisor 256
> >>>/sbin/tc filter add dev eth0 parent 1:0 protocol ip prio 2 u32 ht 2: match
> >>>ip dst 100.64.0.0/19  hashkey mask 0x0000ff00 at 16  link 14:
> >>>
> >>>/sbin/tc filter add dev eth0 parent 1:0 prio 99 handle 170: u32 divisor 256
> >>>/sbin/tc filter add dev eth0 parent 1:0 protocol ip prio 2 u32 ht 14:00:
> >>>match ip dst 100.64.0.0/24 hashkey mask 0x000000ff at 16 link 170:
> >>>
> >>>up to link 19f:
> >>>
> >>>And finally:
> >>>
> >>>/sbin/tc filter add dev eth0 parent 1:0 handle ::2 prio 1 protocol ipv6 u32
> >>>match u32 0 0 link 1:
> >>>
> >>>/sbin/tc filter add dev eth0 parent 1:0 handle ::2 prio 2 protocol ip u32
> >>>match u32 0 0 link 2:
> >>>
> >>>/sbin/tc filter add dev eth0 parent 1:0 prio 3 protocol arp  u32 match u32 0
> >>>0 classid 1:fffe
> >>>
> >>># Rede "default" para endereços IPV6 não Classificados anteriormente
> >>>/sbin/tc filter add dev eth0 parent 1:0 prio 1 handle ::ffe protocol ipv6
> >>>u32 match u32 0 0 classid 1:fffc
> >>>
> >>>Thanks for your time.
> >>>
> >>>Cheers
> >>>
> >>>Ethy
> >>
> >>-- 
> >>Best regards
> >>Anatoly Muliarski
> >
> >



[Index of Archives]     [LARTC Home Page]     [Netfilter]     [Netfilter Development]     [Network Development]     [Bugtraq]     [GCC Help]     [Yosemite News]     [Linux Kernel]     [Fedora Users]
  Powered by Linux