Ethan John wrote: > For the record, setting rr_min_io to something extremely large (we're using > 2 billion now, since I'm assuming it's a C integer) solves the immediate > problem that we're having (overhead in path switching causing poor > performance). Telling people to use mpath_prio_random is still less than > ideal for any small number of iSCSI targets, but it a better short-term > solution for us than nothing. > In setting rr_min_io to something extremely large you effectively disable the round-robin scheduler in multipathing. That's okay for the failover scenario you have (as you only have one path per group), but whenever you have more than one path in a group that wouldn't work anymore. > On 8/10/07, Ethan John <ethan.john@xxxxxxxxx> wrote: >> Hannes, thanks again for your help with this. >> >> I haven't noticed that failback does the right thing, but I'll try it out >> again. Could be something we're doing wrong. In any case, there's very >> little documentation on all this, and I'm trying to develop some kind of >> strategy for our Linux customers to use until we get ALUA implemented. >> >> Being able to set path priorities manually would be ideal, but it seems >> like this is impossible, right? >> >> Here's the situation we have right now. I initiate two connections to one >> target, across two sessions with two different IPs, with two LUs. Multipath >> looks like this: >> mpath45 (20002c9020020001a00151b6b46bb57b0) dm-1 company,iSCSI target >> [size=15G][features=0][hwhandler=0] >> \_ round-robin 0 [prio=1][active] >> \_ 22:0:0:1 sdc 8:32 [active][ready] >> \_ round-robin 0 [prio=1][enabled] >> \_ 23:0:0:1 sde 8:64 [active][ready] >> mpath44 (20002c9020020001200151b6b46bb57ae) dm-0 company,iSCSI target >> [size=15G][features=0][hwhandler=0] >> \_ round-robin 0 [prio=1][enabled] >> \_ 22:0:0:0 sdb 8:16 [active][ready] >> \_ round-robin 0 [prio=1][enabled] >> \_ 23:0:0:0 sdd 8:48 [active][ready] >> >> Note that there are only two active sessions: >> # iscsiadm -m session >> tcp: [20] 10.53.152.22:3260 ,1 iqn.2001-07.com.company:qaiscsi2:blah1 >> tcp: [21] 10.53.152.23:3260,2 iqn.2001-07.com.company:qaiscsi2:blah1 >> >> So the result is that all activity is routed to the first session that was >> initiated. I want to change the priorities of the paths to allow for traffic >> to go to the first IP for mpath45 and the second IP for mpath46. >> That's a matter of the IP routing. Having both target on the same (sub-) net doesn't work very well with multipathing. Please setup your system with each iSCSI Target port in a different subnet eg 10.53.152.22:3260,1 iqn.2001-07.com.company:qaiscsi2:blah1 10.53.153.22:3260,2 iqn.2001-07.com.company:qaiscsi2:blah1 then you'll have one iSCSI target port per subnet and you can actually do failover etc. >> Obviously ALUA is the way to go for this in the future, but we won't have >> the resources to implement that, so I'm looking for an interim solution that >> will scale to thousands of clients. Right now, the only thing I can tell >> people is to manually initiate connections to certain targets through >> certain IP addresses -- basically, doing the load balancing themselves. Is >> there a better way? >> No, not really. But I'm not a network guru. You may want to ask on the open-iscsi mailing list. And you can get all information you need via sysfs, so it should be possible to create a script like Stefan Bader suggested. Cheers, Hannes -- Dr. Hannes Reinecke zSeries & Storage hare@xxxxxxx +49 911 74053 688 SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Markus Rex, HRB 16746 (AG Nürnberg) -- dm-devel mailing list dm-devel@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/dm-devel
