Hello everybody, I have a problem which is similar: 450 customers with 450 HTB classes and their corresponding 450 filters in a subclass (1:3) of the root qdisc (PRIO) of the interface. The problem is manifesting at times, when I try to ping a host behind the router from a host before the router, the latency becomes 1-1,5 msec. On the machine is running also iptables firewall with a bunch of rules for dropping/accepting/natting specific traffic, plus routing about 30 Mbits/sec. When I remove the HTB qdisc, the latency is normal, 0,3-0,4 msec. Anyone has an idea what could cause this? Any input much appreciated! Regards, Stoimen -------------- Pablo, Here we have HTB being used for more than 10.000 customers. The difference, is that we use tc and u32 filters to classify the packets.. I use the same Dell PE 1850, but I have two Quad-Core Xeon (1.86GHz) on it :) # uptime 13:18:08 up 16 days, 12:32, 1 user, load average: 0.02, 0.02, 0.00 mpstat says: 01:19:11 PM CPU %user %nice %sys %iowait %irq %soft % steal %idle intr/s 01:19:13 PM all 0.00 0.00 0.00 0.00 0.57 13.81 0.00 85.61 10568.88 And as you can see.. the use of cpu is not that big.. On May 26, 2007, at 6:54 AM, Pablo Fernandes Yahoo wrote: > Hello, > > > > have HTB „rules“ in 4 different ISPs and i control for each > customer this way: > > > > Flush and 1:0 class > > tc qdisc del dev eth0 root > > tc qdisc add dev eth0 root handle 1:0 htb > > tc class add dev eth0 parent 1:0 classid 1:1 htb rate 100mbit > > tc qdisc del dev eth1 root > > tc qdisc add dev eth1 root handle 1:0 htb > > tc class add dev eth1 parent 1:0 classid 1:1 htb rate 100mbit > > > > Upload and Download: user1 > > tc class add dev eth0 parent 1:1 classid 1:5 htb rate 150kbit ceil > 150kbit > > tc qdisc add dev eth0 parent 1:5 handle 5: sfq perturb 10 > > tc class add dev eth1 parent 1:1 classid 1:5 htb rate 50kbit ceil > 50kbit > > tc qdisc add dev eth1 parent 1:5 handle 5: sfq perturb 10 > > iptables -t mangle -A POSTROUTING --dest x.x.x.x -o eth0 -j > CLASSIFY --set-class 1:5 > > iptables -t mangle -A FORWARD --src x.x.x.x -o eth1 -j CLASSIFY -- > set-class 1:5 > > > > Upload and Download: user2 > > tc class add dev eth0 parent 1:1 classid 1:8 htb rate 150kbit ceil > 150kbit > > tc qdisc add dev eth0 parent 1:8 handle 8: sfq perturb 10 > > tc class add dev eth1 parent 1:1 classid 1:8 htb rate 50kbit ceil > 50kbit > > tc qdisc add dev eth1 parent 1:8 handle 8: sfq perturb 10 > > iptables -t mangle -A POSTROUTING --dest y.y.y.y -o eth0 -j > CLASSIFY --set-class 1:8 > > iptables -t mangle -A FORWARD --src y.y.y.y -o eth1 -j CLASSIFY -- > set-class 1:8 > > > > (…) > > > > This rules works fine, but just for less than 1.700 customers. More > than 1.700 customers, i have my load avarage in the sky and > Ksoftirqd process (top information) in 100% fulltime. I don’t know > why. I used to use CBQ instead HTB because i had the same problem > and Ron (a guy in this list) gave this rules and told me that he > uses this for more than 3.000 customers. I tested it in more than 7 > different computers (but the same hadware specifications) and i had > the same problem with either CBQ or HTB rules. The computers that i > have are all of them DELL PowerEdge 1850. I will put some hardware > iformations here: > > > > top > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > > 3 root 39 19 0 0 0 R 100 0.0 5316:20 > ksoftirqd/0 > > > > [root at fw ~]# uptime > > 10:38:11 up 161 days, 17:21, 3 users, load average: 1.58, 1.65, > 1.51 (unfortunately when i took this, the load average was > „pretty good“, but minutes ago, it was more than 11.0 > > > > [root at fw ~]# lspci > > 00:00.0 Host bridge: Intel Corporation E7520 Memory Controller Hub > (rev 09) > > 00:02.0 PCI bridge: Intel Corporation E7525/E7520/E7320 PCI Express > Port A (rev 09) > > 00:04.0 PCI bridge: Intel Corporation E7525/E7520 PCI Express Port > B (rev 09) > > 00:05.0 PCI bridge: Intel Corporation E7520 PCI Express Port B1 > (rev 09) > > 00:06.0 PCI bridge: Intel Corporation E7520 PCI Express Port C (rev > 09) > > 00:1d.0 USB Controller: Intel Corporation 82801EB/ER (ICH5/ICH5R) > USB UHCI Controller #1 (rev 02) > > 00:1d.1 USB Controller: Intel Corporation 82801EB/ER (ICH5/ICH5R) > USB UHCI Controller #2 (rev 02) > > 00:1d.2 USB Controller: Intel Corporation 82801EB/ER (ICH5/ICH5R) > USB UHCI Controller #3 (rev 02) > > 00:1d.7 USB Controller: Intel Corporation 82801EB/ER (ICH5/ICH5R) > USB2 EHCI Controller (rev 02) > > 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev c2) > > 00:1f.0 ISA bridge: Intel Corporation 82801EB/ER (ICH5/ICH5R) LPC > Interface Bridge (rev 02) > > 00:1f.1 IDE interface: Intel Corporation 82801EB/ER (ICH5/ICH5R) > IDE Controller (rev 02) > > 01:00.0 PCI bridge: Intel Corporation 80332 [Dobson] I/O processor > (A-Segment Bridge) (rev 06) > > 01:00.2 PCI bridge: Intel Corporation 80332 [Dobson] I/O processor > (B-Segment Bridge) (rev 06) > > 02:0c.0 Ethernet controller: Intel Corporation 82545GM Gigabit > Ethernet Controller (rev 04) > > 02:0e.0 RAID bus controller: Dell PowerEdge Expandable RAID > controller 4 (rev 06) > > 03:0b.0 Ethernet controller: Intel Corporation 82545GM Gigabit > Ethernet Controller (rev 04) > > 05:00.0 PCI bridge: Intel Corporation 6700PXH PCI Express-to-PCI > Bridge A (rev 09) > > 05:00.2 PCI bridge: Intel Corporation 6700PXH PCI Express-to-PCI > Bridge B (rev 09) > > 06:07.0 Ethernet controller: Intel Corporation 82541GI/PI Gigabit > Ethernet Controller (rev 05) > > 07:08.0 Ethernet controller: Intel Corporation 82541GI/PI Gigabit > Ethernet Controller (rev 05) > > 09:0d.0 VGA compatible controller: ATI Technologies Inc Radeon > RV100 QY [Radeon 7000/VE] > > > > [root at fw ~]# free -m > > total used free shared buffers > cached > > Mem: 2021 1479 542 0 > 400 654 > > -/+ buffers/cache: 424 1597 > > Swap: 1027 0 1027 > > > > [root at fw ~]# cat /proc/cpuinfo > > processor : 0 > > vendor_id : GenuineIntel > > cpu family : 15 > > model : 4 > > model name : Intel(R) Xeon(TM) CPU 3.00GHz > > stepping : 3 > > cpu MHz : 2992.674 > > cache size : 2048 KB > > physical id : 0 > > siblings : 2 > > core id : 0 > > cpu cores : 1 > > fdiv_bug : no > > hlt_bug : no > > f00f_bug : no > > coma_bug : no > > fpu : yes > > fpu_exception : yes > > cpuid level : 5 > > wp : yes > > flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr > pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm > pbe nx lm constant_tsc pni monitor ds_cpl cid cx16 xtpr > > bogomips : 5990.78 > > > > processor : 1 > > vendor_id : GenuineIntel > > cpu family : 15 > > model : 4 > > model name : Intel(R) Xeon(TM) CPU 3.00GHz > > stepping : 3 > > cpu MHz : 2992.674 > > cache size : 2048 KB > > physical id : 0 > > siblings : 2 > > core id : 0 > > cpu cores : 1 > > fdiv_bug : no > > hlt_bug : no > > f00f_bug : no > > coma_bug : no > > fpu : yes > > fpu_exception : yes > > cpuid level : 5 > > wp : yes > > flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr > pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm > pbe nx lm constant_tsc pni monitor ds_cpl cid cx16 xtpr > > bogomips : 5985.13 > > > > > > Any help/Tipp/hint will be very welcome. > > > > Thanks in Advance! > > > > Pablo Fernandes --------------------------------------------------- Webmail of Bulsat Ltd. at http://mail.bulsattv.com/ _______________________________________________ LARTC mailing list LARTC@xxxxxxxxxxxxxxx http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
