after I changed default_cmdsn_depth to 64 I use iomter to do READ, only core0 is busy, for WRITE, all cores(12 of them) are equally busy. I created 12 target(each has one LUN) for 12-cores in this case, still the performance for both READ and WRITE are about 1/3 comparing to SCST I got in the past. is LIO-iSCSI on 3.8.x 'best' for 10/100/1G network only? other than the DEFAULT_CMDSN_DEPTH definition what else I could tune for 10G/40G iSCSI? Again I am using the same scheduler/ fifo_batch strip_cache_size read_ahead_kb etc parameters as I used with SCST, the only major difference is LIO vs SCST itself. Thanks... On Wed, Oct 2, 2013 at 12:12 PM, Xianghua Xiao <xiaoxianghua@xxxxxxxxx> wrote: > On Wed, Oct 2, 2013 at 11:22 AM, Xianghua Xiao <xiaoxianghua@xxxxxxxxx> wrote: >> This is indeed a 40G user case. Thanks for the info on >> default_cmdsn_depth, it's set to 16 on 3.8.x kernel and I am changing >> it to 64. >> >> How can I enable MCS and/or I_T nexus? At the moment I want to see how >> fast the stack can handle iSCSI/RAID5 throughput. >> >> Thanks, >> xiao >> >> >> On Tue, Oct 1, 2013 at 4:16 PM, Nicholas A. Bellinger >> <nab@xxxxxxxxxxxxxxx> wrote: >>> On Wed, 2013-10-02 at 01:14 +0800, Xianghua Xiao wrote: >>>> I used lio-utils to get iscsi working, however during testing I found >>>> out that only 2 out of 12 cores are being used by LIO-TARGET(iblock), >>>> why is that? >>>> >>>> Used SCST in the past and the workload was shared by all the cores by >>>> default, so it's a surprise to me with LIO-iscsi that it picks one >>>> core for Tx and one core for Rx, the rest cores are sitting idle? >>>> >>>> any help is appreciated. >>>> >>> >>> Two items here. >>> >>> First, you'll want to bump the per TPG attribute value for >>> default_cmdsn_depth from the default 16 to something larger (64 or 128). >>> >>> This value controls how many outstanding commands the initiator is >>> allowed to dispatch on a per session basis, and increasing the value >>> allows a larger pipe (10 Gb/sec for example) to keep more active >>> commands in flight. The default value has also been increased to a >>> higher value starting in v3.12-rc1 code in the patch here: >>> >>> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/drivers/target/iscsi?id=38f7d6edbf91693bb679bb29901c8296e7838cd9 >>> >>> Second, in order to scale the number of threads used for I/O submission, >>> you'll either want to enable MC/S (multiple connections per session), or >>> use multiple sessions (typically across multiple ports) with multipath >>> on the client side. >>> >>> --nab >>> -- To unsubscribe from this list: send the line "unsubscribe target-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
