Your numbers seem reasonable for a 700MHz system. Packet forwarding rates will depend on your device driver and hardware. Look for drivers with NAPI (polling) support to get best numbers. I measured forwarding rates of around 100kpps with a 1GHz Pentium-III system and Intel Pro100 ethernets using the latest e100-v3 device driver and a 2.4 kernel. Rates increased a little (to 102kpps in my case) with 2.6 kernels, mostly due to fastpath improvements in the networking/kernel core. NAPI (polling) significantly reduces interrupt overhead when under load, whilst using interrupts to "wake up" the device driver to begin polling when it is idle. Under heavy loads, device interrupts are turned off by the NAPI driver so the kernel decides what events to schedule when, not the CPU and network interface's interrupt hardware. The livelock issue you mention occurs when drivers do too much work in their irq handlers when there is no mechanism to throttle interrupt rate. Since interrupt handling is done at highest "priority" above any other kernel work, the CPU does nothing other than handle interrupts. Note that some NAPI-enabled drivers use NAPI polling only for receive processing; they leave transmit interrupts enabled which are handled directly in the irq handler. (I think this was the original NAPI design.) I found that even better forwarding performance is achieved by doing rx _and_ tx work in polled mode, thus leaving device interrupts disabled as much as possible. The latest e100-v3 driver has this optimization. The performance improvement is very noticeable during bursts of tx traffic, since NAPI is enabled even when there is no rx traffic. This behavior is ideal for network forwarding devices, where traffic bursts can be in one direction. Some devices support hardware interrupt mitigation, where the hardware throttles max interrupt rate, generating one interrupt per N events or on a h/w timer. Chip manufacturers often make a big deal out of these sorts of features. With Linux, however, I've never seen h/w interrupt mitigation yield better forwarding performance than a tx/rx-NAPI-enabled driver with h/w interrupt mitigation features turned off... Unfortunately, few Linux network drivers support NAPI. If your hardware has already been chosen and the driver doesn't yet support NAPI, add NAPI support to get best performance. See Documentation/networking/NAPI_HOWTO.txt in the kernel sources for instructions on how to do so. -JC -----Original Message----- From: Surya Srinivasan [mailto:surya_srinivasan@hotmail.com] Sent: 01 June 2004 00:28 To: linux-net@vger.kernel.org Subject: Linux router forwarding performance We are planning to use a linux PC as a router in our lab. I was wondering if anyone has recently measured the forwarding rate using a standard PC (say Pentium)? Have there been any changes to the kernel to improve the forwarding rate? The best performance I have seen so far is with the Click modular router ( http://www.pdos.lcs.mit.edu/papers/click:tocs00/paper.pdf ). According to their study on a 700Mhz Pentium (64 byte packets) Linux does 75kpps Linux polling 300kpps (Polling amortizes the device handling overhead over several packets when packet rates are high is my understanding. It also avoids the receive livelock issue). Is receive livelock still an issue with Linux today? Or is this avoided by throttling the receive interfaces? Is there anyone using polling mode for better forwarding rates? I have been looking for documentation in this regard and would appreciate any pointers. If there is a more appropriate list for such questions kindly point me to the right list. thanks Surya _________________________________________________________________ Is your PC infected? Get a FREE online computer virus scan from McAfeeR Security. http://clinic.mcafee.com/clinic/ibuy/campaign.asp?cid=3963 - : send the line "unsubscribe linux-net" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html - : send the line "unsubscribe linux-net" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
