On Thu, Jun 25 2015 at 12:07pm -0400, Derick Swanepoel <dswanepoel@xxxxxxxxx> wrote: > Hi > > We have an iSCSI-based SAN that uses dm-multipath to aggregate two 10 Gb links in multibus mode. With kernel 3.11.6 we got very good performance, achieving up to 1550 MB/s read on the multipath device. > > After upgrading to kernel 4.0.5, performance has dropped by 40%. Read speed is now limited to 960 MB/s, which is slower than the 975 MB/s that the individual paths can achieve. > > We did a lot of testing on 3.11.6 to find an optimal rr_min_io_rq value and settled on 9 using the round-robin path selector. With 4.0.5, we’ve tried all the path selectors with rr_min_io_rq from 1 to 100, but can’t seem to beat the performance of a single path any more. For interest’s sake, the default rr_min_io_rq of 1 was actually one of the worst performing values. > > We also tried kernel 3.12 and performance was also degraded but not quite as bad as 4.0.5. I am wondering if there was some significant change since 3.11 (dm-multipath 1.5.1) and 3.12 (dm-multipath 1.6.0). Nothing stands out to me: # git log --oneline --tags --source drivers/md/dm.c drivers/md/dm-mpath.c | grep 3.12- | tac 670368a dm-3.12-changes dm: stop using WQ_NON_REENTRANT 7e782af v3.12-rc1 [SCSI] Return ENODATA on medium error 00c4fc3 dm-3.12-changes dm ioctl: increase granularity of type_lock when loading table fd2ed4d dm-3.12-changes dm: add statistics support 7426d62 v3.12-rc1 Merge tag 'dm-3.12-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm cc9d3c3 dm-3.12-fixes dm mpath: do not fail path on -ENOSPC f84cb8a dm-3.12-fixes dm mpath: disable WRITE SAME if it fails 6cfa585 dm-3.12-fixes dm: lower bio-based mempool reservation f479082 dm-3.12-fixes dm: add reserved_rq_based_ios module parameter e860313 dm-3.12-fixes dm: add reserved_bio_based_ios module parameter -- dm-devel mailing list dm-devel@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/dm-devel
