On Tue, 2005-05-24 at 21:28 -0400, open_iscsi wrote: > But it is not multi-pathing. Multi-pathing belongs at a higher layer. > > Yes, you could make multi-pathing perform a similar action but being at a > higher layer, it means more operations to achieve the same thing. Also, > multi-pathing is better suited for failover than multi-connections. > > There is another point here ... an HBA will probably use multi-connections > irrespective of what higher layers want. > > Regarding the numbers, we get 400,000 IOPS with our hardware solution using > multiple connections and multiple micro-engines. This number is impressive. I can not believe it is on Initiator side since quite a bit of code are involved besides TCP/IP: userspace app, VFS, SCSI-ML and LLDD (even though iSCSI HBA can do zero-copy on receive). With open-iscsi/linux-iscsi-5.x on very fast hardware the best we could get is 75,000 IOPS. And we believe it is a world record among other iSCSI software initiators. I also did comparison between multipath-like and MC/S-like setups and found that multipath-like setup scales much better, especially for WRITE's we found that scale factor is ~1.75. I.e. with single session we've got ~500MB/sec throughput and with two sessions we've got ~800MB/sec. > I have not tried > multi-pathing but I can tell you that I had to count clocks to get that > number and found that even a few extra clocks could mean a lot. So since > multi-pathing takes a lot of extra clocks, then I think there is a benefit. > However with a software solution the extra clocks for the multi-pathing may > not be significant. > > I would think that you would want to let the lower layers do their best to > get the best thruput and leave the failover logic to the upper layers. > > Eddy > > ----- Original Message ----- > From: "James Bottomley" <James.Bottomley@xxxxxxxxxxxx> > To: "open_iscsi" <ESQuicksall_open_iscsi@xxxxxxxxxxx> > Cc: <open-iscsi@xxxxxxxxxxxxxxxx>; "Mike Christie" <michaelc@xxxxxxxxxxx>; > "'SCSI Mailing List'" <linux-scsi@xxxxxxxxxxxxxxx> > Sent: Tuesday, May 24, 2005 9:00 PM > Subject: Re: [PATCH RFC 2/2] implement transport scan callout for iscsi > > > > On Tue, 2005-05-24 at 20:25 -0400, open_iscsi wrote: > >> The MC/S feature of iSCSI is not multi-pathing. Multi-pathing would be > >> the > >> use of multiple sessions to reach the same target. Generally the two > >> sessions would use the same InitiatorName+ISID but use different Target > >> Portal Groups at the target. In SCSI terms, it is the same initiator > >> accessing different SCSI ports. > > > > Well, yes, every driver vendor with a multi-path solution in-driver that > > made a single presentation to the mid-layer has argued that one... > > > > The bottom line is that implementation must be in-driver. So every > > driver doing it this way has to have their own separate multi-path > > implementation. Whether you call it FC/AL or MC/S (or any of the other > > buzz acronyms) it's still a driver implementation of pathing. > > > >> MC/S can be used to improve band width of a session without using > >> multi-pathing and it belongs in the driver because it is hidden from the > >> upper layers. Think of it like parallel wires, each carrying separate > >> (but > >> sequenced) commands in parallel. > > > > So far, no-one has been able to produce any figures to show that MC/S is > > significantly better than symmetric active dm-multipath to an iSCSI > > target, but if you have them, please publish them. > > > > Hiding something from the upper layers which the upper layers could do > > equally well themselves is what's considered wrong: it adds code bloat > > with no tangible benefit. > > > > James > > > > > - : send the line "unsubscribe linux-scsi" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
