Michael, First, I don't know if my personal findings are applicable to your situation. I am using much older kernels, so the issue I ran into may be obsolete or at least things may be different. I found that sometimes when sending network packets, the kernel would hijack the thread of the sender when the queue was empty. This would cause issues, since I had no control over the priority of that thread, since it could be anything. My fix was to never allow the kernel to send in the context of the sending userspace thread. I suspect that most uses of the real time kernel don't expect the real time nature of things to extend across the network, but I could be wrong. Best Regards, Dave -----Original Message----- From: linux-rt-users-owner@xxxxxxxxxxxxxxx [mailto:linux-rt-users-owner@xxxxxxxxxxxxxxx] On Behalf Of Michael Hewitt Sent: Monday, March 14, 2016 3:02 PM To: linux-rt-users@xxxxxxxxxxxxxxx Subject: Eliminating Packet Latency Folks, Please forgive me if this is an inappropriate forum for my question. We are using a real time Linux kernel (3.10.0) in a network appliance in order to achieve extremely consistent packet delivery times. Generally, we see packet delivery variations of less than 100 microseconds, which is fabulous. Occasionally, we see a packet delivery delay in excess of 1000 microseconds. We are hoping to eliminate these spikes, which occur perhaps 1-2 times in a 24 hour period. The machine configuration is as follows. Thread IRQs are enabled, and we have elevated the priority of both the irq threads that service the specific network interface to 55. We have also elevated the priority of the relevant user space thread to 49. We are running on a 4 core Intel Xeon E3-1220v3 with an Intel NIC and the igb version 5.3.2 driver. We disabled interrupt throttling in the Intel driver (rx-usecs = 0, tx-usecs = 0). SE Linux is disabled, eliminating a huge packet latency spike during login. We are running CentOS 7.1 tuned for network latency ("tuned-adm profile network-latency"). IRQ balancing is disabled. BIOS CPU power management is set to maximum performance. Any help you can provide would be greatly appreciated. Thanks, Mike -- To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
Attachment:
smime.p7s
Description: S/MIME cryptographic signature
