On Wed, 2009-03-25 at 17:52 +0200, Pasi Kärkkäinen wrote: > On Tue, Mar 24, 2009 at 11:41:00PM -0400, John A. Sullivan III wrote: > > > > Latency seems to be our key. If I can add only 20 micro-seconds of > > > > latency from initiator and target each, that would be roughly 200 micro > > > > seconds. That would almost triple the throughput from what we are > > > > currently seeing. > > > > > > > > > > Indeed :) > > > > > > > Unfortunately, I'm a bit ignorant of tweaking networks on opensolaris. > > > > I can certainly learn but am I headed in the right direction or is this > > > > direction of investigation misguided? Thanks - John > > > > > > > > > > Low latency is the key for good (iSCSI) SAN performance, as it directly > > > gives you more (possible) IOPS. > > > > > > Other option is to configure software/settings so that there are multiple > > > outstanding IO's on the fly.. then you're not limited with the latency (so much). > > > > > > -- Pasi > > <snip> > > Ross has been of enormous help offline. Indeed, disabling jumbo packets > > produced an almost 50% increase in single threaded throughput. We are > > pretty well set although still a bit disappointed in the latency we are > > seeing in opensolaris and have escalated to the vendor about addressing > > it. > > > > Ok. That's pretty big increase. Did you figure out why that happens? Greater latency with jumbo packets. > > > The once piece which is still a mystery is why using four targets on > > four separate interfaces striped with dmadm RAID0 does not produce an > > aggregate of slightly less than four times the IOPS of a single target > > on a single interface. This would not seem to be the out of order SCSI > > command problem of multipath. One of life's great mysteries yet to be > > revealed. Thanks again, all - John > > Hmm.. maybe the out-of-order problem happens at the target? It gets IO > requests to nearby offsets from 4 different sessions and there's some kind > of locking or so going on? Ross pointed out a flaw in my test methodology. By running one I/O at a time, it was literally doing that - not one full RAID0 I/O but one disk I/O apparently. He said to truly test it, I would need to run as many concurrent I/Os as there were disks in the array. Thanks - John > > Just guessing. > > -- Pasi > > -- > dm-devel mailing list > dm-devel@xxxxxxxxxx > https://www.redhat.com/mailman/listinfo/dm-devel -- John A. Sullivan III Open Source Development Corporation +1 207-985-7880 jsullivan@xxxxxxxxxxxxxxxxxxx http://www.spiritualoutreach.com Making Christianity intelligible to secular society -- dm-devel mailing list dm-devel@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/dm-devel
