Simon Lodal wrote: >I have similar hardware, load and trouble. > >Interrupts are only sent to one cpu, instead of all of them, because that was >only overhead. I think the default was changed somewhere around 2.6.10 >or .12, but I have forgotten the url. > >There is a CONFIG_IRQBALANCE option in the kernel, but last time I checked >(2.6.16) it did not work very well; almost never does anything. So I have >turned it off. I use the userspace irqbalance daemon that periodically sets >the smp_affinity's, with the effect that ingoing and outgoing traffic are >handled by each their cpu (assuming that no other interrupts or processes are >significant). That helps some. But we only shape in one direction, and it can >not help spread the shaping load between the CPU's. > >There is also an acpi_irq_balance kernel parameter (not related to the kernel >irq balancer), which apparently uses the APIC to do interrupt round-robin. It >worked surprisingly well (perfect, actually) on an old dual celeron that I >tested; the network interrupts are spread nicely and evenly across the cpu's. >It is probably very chipset dependent, and I have not yet tested it on the >firewalls. > >But as I understand from this: >http://vger.kernel.org/~davem/cgi-bin/blog.cgi/2006/09/14#netconf2006_day2 >it might not even be an advantage, since the current shaping code would just >make the cpu's step on each others toes. > > >Regards, >Simon > > >On Saturday 16 September 2006 20:39, Alexandru Dragoi wrote: > > >>Hello, >> >>Here is the situation. There is a machine with 3 intel gigabit card, 2 >>of them on PCI-X and in bridge, the 3rd is used only for management >>access. The machine is a dual Xeon 2.8GHz with HT. With 2.6.8 kernel >>from debian (testing) and htb with u32 on, i usually get about 30-40% >>software interrupts on CPU0 and CPU2, and without htb and u32, 10% less. >>Now, if I boot with 2.6.17.9 kernel, first all irqs are on same CPU. I >>managed with smp_afinity to "move" irq of one card to a different CPU. >>In these circumstances, I have about 20% or a little less soft >>interrupts on each CPU without shaping, but about 60-70% os soft >>interrupts with shapping, and sometimes there is packet loss, also >>dropped packets are shown on ifconfig. The htb script is same. I have >>u32 performance counters enabled on u32 in 2.6.17.9. I also have NAPI, >>of course. I can't think on anything else that can cause the problem, >>but seem to be something in the kernel. Here is the output of lspci: >> >># lspci >>0000:00:00.0 Host bridge: Intel Corporation E7320 Memory Controller Hub >>(rev 0a) >>0000:00:00.1 ff00: Intel Corporation E7320 Error Reporting Registers >>(rev 0a) >>0000:00:02.0 PCI bridge: Intel Corporation E7525/E7520/E7320 PCI Express >>Port A (rev 0a) >>0000:00:03.0 PCI bridge: Intel Corporation E7525/E7520/E7320 PCI Express >>Port A1 (rev 0a) >>0000:00:1c.0 PCI bridge: Intel Corporation 6300ESB 64-bit PCI-X Bridge >>(rev 02) >>0000:00:1d.0 USB Controller: Intel Corporation 6300ESB USB Universal >>Host Controller (rev 02) >>0000:00:1d.1 USB Controller: Intel Corporation 6300ESB USB Universal >>Host Controller (rev 02) >>0000:00:1d.4 System peripheral: Intel Corporation 6300ESB Watchdog Timer >>(rev 02) >>0000:00:1d.5 PIC: Intel Corporation 6300ESB I/O Advanced Programmable >>Interrupt Controller (rev 02) >>0000:00:1d.7 USB Controller: Intel Corporation 6300ESB USB2 Enhanced >>Host Controller (rev 02) >>0000:00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 0a) >>0000:00:1f.0 ISA bridge: Intel Corporation 6300ESB LPC Interface >>Controller (rev 02) >>0000:00:1f.2 IDE interface: Intel Corporation 6300ESB SATA Storage >>Controller (rev 02) >>0000:00:1f.3 SMBus: Intel Corporation 6300ESB SMBus Controller (rev 02) >>0000:03:02.0 Ethernet controller: Intel Corporation 82544EI Gigabit >>Ethernet Controller (Copper) (rev 02) >>0000:03:03.0 Ethernet controller: Intel Corporation 82544EI Gigabit >>Ethernet Controller (Copper) (rev 02) >>0000:04:02.0 VGA compatible controller: ATI Technologies Inc Rage XL >>(rev 27) >>0000:04:03.0 Ethernet controller: Intel Corporation 82541GI Gigabit >>Ethernet Controller (rev 05) >> >>The traffic is somewhere at 40kpps of traffic and 120mbit up, >>120mbit/down. Suggestions about better hardware and kernel, or links >>with docs about these are really welcomed. >> >>Bye >> >>_______________________________________________ >>LARTC mailing list >>LARTC@xxxxxxxxxxxxxxx >>http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc >> >> Somebody recomended me to use Gigabit Intel dual port and kernel 2.6.9, because some distrubutions stopped at 2.6.9. Now, if things changed on SMP since 2.6.10 or 2.6.12, I believe those distributions (perhaps some Redhat and CentOS) know what are they doing. _______________________________________________ LARTC mailing list LARTC@xxxxxxxxxxxxxxx http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
