On Thu, Oct 3, 2013 at 2:18 PM, Nicholas A. Bellinger <nab@xxxxxxxxxxxxxxx> wrote: > On Thu, 2013-10-03 at 09:16 -0500, Xianghua Xiao wrote: >> The IRQ is balanced to all cores(cat /proc/interrupts), the option is >> turned on via menuconfig. Still the performance is the same. >> > > Please don't top-post. It makes it annoying to respond to what's > already been said in the thread. > > FYI, there is no kernel option to balance IRQs automatically across > CPUs, it's done via userspace using irqbalanced, or via explicit > settings in /proc/irq/$IRQ/smp_affinity_list. I checked /proc/interrupts and verified all cores are getting interrupts. also CONFIG_IRQ_ALL_CPUS=y > > So, I'd still like to see your /proc/interrupts output in order to > determine the distribution. > > Some top and perf top output would be useful as well to see what > processes and functions are running. > >> The emulate_write_cache=1 did not help performance either. >> >> How does LIO/iscsi handle multi-thread on multi-core system? >> > > As explained below: > > So target_core_mod uses a bounded workqueue for it's I/O completion, > which means that process context is provided on the same CPU for which > the hardware interrupt was generated in order to benefit from cache > locality effects. If all of the hardware interrupts for the entire > system are firing only on CPU0, then only kworkerd/0 is used to provide > process context for queuing the response to the fabric drivers. > > Also, I can confirm with v3.11 code that iscsi-target is running dual > port ixgbe line rate (~20 Gb/sec) with large block reads/writes to PCIe > flash, and to ramdisk_mcp backends. > > So that said, I'll need more information about your setup to determine > what's going on. For iSCSI, all cpus are equally busy(verified by 'top') and all cores are getting the same number of interrupts. Sigh, I have to stick with 3.8.x kernel for now, this is a non-x86 box so it's hard to upgrade the kernel due to various dependencies. Thanks! > > --nab > -- To unsubscribe from this list: send the line "unsubscribe target-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
