I have a quad-core Xeon with 4 gigabit NICs. I'm sending gobs of data out three of them (about 93% of capacity). One on machine, identical to the other, I have a problem where CPU0 is 0% idle while the other 3 cores are over 60% idle. My application mostly avoids CPU0, and there is very little user time being spent on CPU0. It is almost all irq (about 25%) and sys (about 50%). On the other machine, there's a little bit more sys and irq time on CPU0, but everything's pretty well balanced. If I kill irqbalance and manually set smp_affinity to avoid CPU0, nothing much changes. /proc/interrupts shows a fair number of interrupts going to CPU0 before I make the change, and none after (other than "LOC" interrupts), but the CPU0 load doesn't change much at all. There are hardly any packets coming into those three NICs; almost everything is a set of (several hundred) MPEG streams going out. I guess my main question is: if /proc/interrupts doesn't show any interrupts going to CPU0, then why is it spending any time on interrupt handling? A related question: does NAPI apply to sending packets as well as receiving them? How can I tell whether (1) my drivers support NAPI and (2) it is active? I've tried playing with systemtap a bit to try to figure out what that CPU0 time is going to, but I don't really know how to use it well enough. Here's a snapshot of 'atop' output showing things when they aren't quite as extreme (CPU0 is 56% sys, 14% irq, 11% user). This is with all the interrupts shunted away from CPU0, or at least, all the ones that would let me. PRC | sys 9.73s | user 9.55s | #proc 839 | #zombie 0 | #exit 182 | CPU | sys 78% | user 96% | irq 46% | idle 170% | wait 10% | cpu | sys 56% | user 11% | irq 14% | idle 19% | cpu000 w 0% | cpu | sys 8% | user 32% | irq 7% | idle 51% | cpu001 w 2% | cpu | sys 7% | user 27% | irq 12% | idle 49% | cpu003 w 5% | cpu | sys 7% | user 26% | irq 13% | idle 51% | cpu002 w 3% | CPL | avg1 20.95 | avg5 20.11 | avg15 20.09 | csw 233459 | intr 204338 | MEM | tot 3.5G | free 214.6M | cache 1.3G | buff 41.5M | slab 38.2M | SWP | tot 2.0G | free 2.0G | | vmcom 3.2G | vmlim 3.7G | DSK | sda | busy 22% | read 0 | write 138 | avio 15 ms | DSK | sdb | busy 21% | read 0 | write 138 | avio 15 ms | NET | transport | tcpi 45226 | tcpo 34604 | udpi 608 | udpo 273 | NET | network | ipi 45834 | ipo 2667927 | ipfrw 0 | deliv 45834 | NET | eth2 96% | pcki 2 | pcko 883468 | si 0 Kbps | so 962 Mbps | NET | eth3 95% | pcki 1 | pcko 876330 | si 0 Kbps | so 954 Mbps | NET | eth1 95% | pcki 1 | pcko 872871 | si 0 Kbps | so 951 Mbps | NET | lo ---- | pcki 22099 | pcko 22099 | si 1532 Kbps | so 1532 Kbps | NET | eth0 0% | pcki 23426 | pcko 12763 | si 2948 Kbps | so 1557 Kbps | # uname -r 2.6.18-8.el5.tvh.3 (preemption is enabled) # rpm -q centos-release centos-release-5-0.0.el5.centos.2 # egrep 'proc|model name' /proc/cpuinfo processor : 0 model name : Intel(R) Xeon(R) CPU E5472 @ 3.00GHz processor : 1 model name : Intel(R) Xeon(R) CPU E5472 @ 3.00GHz processor : 2 model name : Intel(R) Xeon(R) CPU E5472 @ 3.00GHz processor : 3 model name : Intel(R) Xeon(R) CPU E5472 @ 3.00GHz # lspci | fgrep Eth 03:00.0 Ethernet controller: Broadcom Corporation Unknown device 165a 04:00.0 Ethernet controller: Broadcom Corporation Unknown device 165a 09:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (rev 06) 09:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (rev 06) # ethtool -i eth1 driver: tg3 version: 3.65-rh firmware-version: 5722-v3.07 bus-info: 0000:04:00.0 # ethtool -i eth2 driver: e1000 version: 7.2.7-k2-NAPI firmware-version: 5.11-2 bus-info: 0000:09:00.0 # ethtool -i eth3 driver: e1000 version: 7.2.7-k2-NAPI firmware-version: 5.11-2 bus-info: 0000:09:00.1 -- To unsubscribe from this list: send the line "unsubscribe linux-net" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
