On Mon, Mar 20, 2023 at 03:18:37PM +0100, Martin Wilck wrote: > On Thu, 2023-03-16 at 14:47 -0700, Brian Bunker wrote: > > As a target vendor, it is nice to be able control initiator > > behavior from the target without relying on user intervention > > on the initiator. There could be a very large number of initiators > > at a site. > > > > When ACLs are first added for a volume on our array, we use the > > transport layer, so that the initiator will discover the volumes > > without any manual intervention. > > > > kernel: scsi 8:0:0:1: Direct-Access PURE Flash Array > > 8888 PQ: 0 ANSI: 6 > > kernel: scsi 9:0:0:1: Direct-Access PURE Flash Array > > 8888 PQ: 0 ANSI: 6 > > kernel: scsi 6:0:0:1: Direct-Access PURE Flash Array > > 8888 PQ: 0 ANSI: 6 > > kernel: scsi 7:0:0:1: Direct-Access PURE Flash Array > > 8888 PQ: 0 ANSI: 6 > > ... > > kernel: sd 6:0:0:1: [sdd] Attached SCSI disk > > kernel: sd 8:0:0:1: [sdb] Attached SCSI disk > > kernel: sd 9:0:0:1: [sdc] Attached SCSI disk > > kernel: sd 7:0:0:1: [sde] Attached SCSI disk > > > > Subsequent volumes after the first one are discovered via unit > > attentions triggering the udev rule which calls scan-scsi-target. > > The SCSI devices being discovered without creating the corresponding > > multipath devices seems to be a bad default. We would like to > > control as much as possible from the target side to dictate initiator > > behavior. This comes as a regression to how it previously worked. > > > > Signed-off-by: Brian Bunker <brian@xxxxxxxxxxxxxxx> > > I'm fine with this, but keep in mind that distributions will probably > override this anyway. Red Hat and SUSE have had different defaults for > this basically forever. At least enterprise distros won't risk > regressions because of changing defaults. > > Ben, what's your opinion wrt the patch? tl;dr: I think "yes" makes more sense than "smart". I don't know if this is a good idea. The default behavior we set is going to be what happens when people don't set up a configuration file. I get that "strict" means you have to manually set up maps. But that actually seems like a reasonable default if you don't have a configuration set up. Using "no" or "greedy" means that you have to set up a configuration, or multipath will just use all your devices, and that seems much worse. But if we want to make multipath "just do the right thing without getting in my way", then I would argue "yes" is a better alternative. The benefit of using "yes" is that multipath will almost always correctly find your multipath devices, and will never fail in a way where it grabs devices it shouldn't. The only time it will fail is on a multipathable device that has never been multipathed before, and it will only not work if something else starts using the first path of the device before the second path appears and multipathd creates a multipath device on it. This really only happens when this new device has some metadata on it that causes something to automatically grab it (for instance a LABEL for a filesystem that gets automounted, or LVM metadata for a device that gets autoactivated). I don't actually know of any real downsides to using "yes", and if there were some, they would also be downsides to using "smart" There are real downsides to using "smart" without setting up a configuration file. Every single time you boot, the rest of the system's access to your possibly multipathable devices is delayed while multipath waits seconds for a sibling to appear. In return for this issue that happens on every boot for every possibly multipathable device, the only benfit you get over "yes", is that when you add a new device to your system, if there is data on the device that would cause it to be autoassembled and the second path appears within seconds of the first path (either 1 or 10, depending on whether or not there is a built-in config for the storage array), multipath will correctly grab the device, instead of whatever was going to autoassemble on it. This is a very rare occurance, still leaves you with a running system, and can easily be fixed after the fact. There's only one time when RHEL makes use of "smart", and that's during installation. For reasons which I don't understand, the RHEL installer will autoassemble LVM/MD devices if there is existing metadata on disks when it boots. In this case the system is unavoidably seeing all of the storage devices for the very first time, without multipath being configured for these devices, and it not unlikely that we will see devices with LVM/MD metadata on them. This means that LVM/MD will likely autoassemble before the second path appears, and the device gets multipathed. This confuses the installer. Since we only do this in the installer, we only see the "smart" delay on releasing the devices to the systme one time. In this situation, using "smart" makes sense (although not as much sense as simply not autoassembling LVM devices when the installer boots, IMHO). The only other situation where "smart" would be generally helpful is if you have your system configured so that all devices are blacklisted except the types that are supposed to be multipathed. In this situation you wouldn't have to worry about the delay on every boot because all the non-multipathable devices would be blacklisted. If a new multipathable device appeared, then "smart" would guarantee that nothing else would grab it before the second path appeared (assuming that the second path appeared within the timeout). However, you quite likely still shouldn't use "smart" in this case. If you already have your configuration set up like this, then you can just use "greedy" and get the same benefit, without having to worry about the second path showing up on time.\ It is possible that you can't set up you configuration to correctly sort all the devices that may appear in the future into multipathable and non-multipathable. In this case, if it's important that these new devices are correctly multipathed the first time they show up, then "smart" also makes sense. But I don't think that this case was so common that we should assume that it's the default for people who install the multipath tools. It takes very little effort to change the find_multipaths setting. The people who aren't interested enough in their multipath setup to do that probably aren't the people that want multipath claiming their devices for a couple seconds every boot, just in case we're in that rare situation where it could make a difference. -Ben > Regards > Martin -- dm-devel mailing list dm-devel@xxxxxxxxxx https://listman.redhat.com/mailman/listinfo/dm-devel
