On Tue, Mar 24, 2009 at 08:21:45AM -0400, John A. Sullivan III wrote: > > > > Core-iscsi developer seems to be active developing at least the > > new iSCSI target (LIO target).. I think he has been testing it with > > core-iscsi, so maybe there's newer version somewhere? > > > > > We did play with the multipath rr_min_io settings and smaller always > > > seemed to be better until we got into very large numbers of session. We > > > were testing on a dual quad core AMD Shanghai 2378 system with 32 GB > > > RAM, a quad port Intel e1000 card and two on-board nvidia forcedeth > > > ports with disktest using 4K blocks to mimic the file system using > > > sequential reads (and some sequential writes). > > > > > > > Nice hardware. Btw are you using jumbo frames or flow control for iSCSI > > traffic? > > Dunno if you noticed this.. :) > > > > > > > When you used dm RAID0 you didn't have any multipath configuration, right? > Correct although we also did test successfully with multipath in > failover mode and RAID0. > > OK. > > What kind of stripe size and other settings you had for RAID0? > Chunk size was 8KB with four disks. > > Did you try with much bigger sizes.. 128 kB ? > > What kind of performance do you get using just a single iscsi session (and > > thus just a single path), no multipathing, no DM RAID0 ? Just a filesystem > > directly on top of the iscsi /dev/sd? device. > Miserable - same roughly 12 MB/s. OK, Here's your problem. Was this btw reads or writes? Did you tune readahead-settings? Can paste your iSCSI session settings negotiated with the target? > > > > Sounds like there's some other problem if invidual throughput is bad? Or did > > you mean performance with a single disktest IO thread is bad, but using multiple > > disktest threads it's good.. that would make more sense :) > Yes, the latter. Single thread (I assume mimicking a single disk > operation, e.g., copying a large file) is miserable - much slower than > local disk despite the availability of huge bandwidth. We start > utilizing the bandwidth when multiplying concurrent disk activity into > the hundreds. > > I am guessing the single thread performance problem is an open-iscsi > issue but I was hoping multipath would help us work around it by > utilizing multiple sessions per disk operation. I suppose that is where > we run into the command ordering problem unless there is something else > afoot. Thanks - John You should be able to get many times the throughput you get now.. just with a single path/session. What kind of latency do you have from the initiator to the target/storage? Try with for example 4 kB ping: ping -s 4096 <ip_of_the_iscsi_target> 1000ms divided by the roundtrip you get from ping should give you maximum possible IOPS using a single path.. 4 kB * IOPS == max bandwidth you can achieve. -- Pasi -- dm-devel mailing list dm-devel@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/dm-devel
