On Fri, Jun 9, 2023 at 12:23 AM Bart Van Assche <bvanassche@xxxxxxx> wrote: > > On 6/7/23 19:51, Jianlin Lv wrote: > > On Thu, Jun 8, 2023 at 1:07 AM Bart Van Assche <bvanassche@xxxxxxx> wrote: > >> On 6/7/23 08:55, Jianlin Lv wrote: > >> I see two possible solutions: > >> - Change the volume provisioner such that it uses disk references that do > >> not depend on the probing order, e.g. /dev/disk/by-id/... > > > > Yes, The "/dev/disk/by-id/" can uniquely identify SCSI devices. However, > > I don't think it is suitable for the volume provisioner workflow. > > For nodes of the same SKU , a unified YAML file will be defined to instruct > > the volume provisioner on how to manage the local disks. > > If use WWID, it would mean that a unique YAML file needs to be defined > > for each node. This approach becomes impractical when dealing with a large > > number of work nodes. > Please consider using the paths available in /dev/disk/by-path. Sorry for the late reply. I carefully checked the server in the production environment and found some corner cases where there are differences in the dev/disk/by-path/ of nodes with the same SKU. These differences are caused by inconsistent target_numbers. For example: diff -y aa-by-path bb-by-path pci-0000:86:00.0-scsi-0:3:86:0 -> ../../sda | pci-0000:86:00.0-scsi-0:3:88:0 -> ../../sda pci-0000:86:00.0-scsi-0:3:87:0 -> ../../sdb | pci-0000:86:00.0-scsi-0:3:89:0 -> ../../sdb pci-0000:86:00.0-scsi-0:3:88:0 -> ../../sdc | pci-0000:86:00.0-scsi-0:3:90:0 -> ../../sdc pci-0000:86:00.0-scsi-0:3:89:0 -> ../../sdd | pci-0000:86:00.0-scsi-0:3:91:0 -> ../../sdd pci-0000:86:00.0-scsi-0:3:90:0 -> ../../sde | pci-0000:86:00.0-scsi-0:3:92:0 -> ../../sde pci-0000:86:00.0-scsi-0:3:92:0 -> ../../sdf | pci-0000:86:00.0-scsi-0:3:93:0 -> ../../sdf pci-0000:86:00.0-scsi-0:3:93:0 -> ../../sdg | pci-0000:86:00.0-scsi-0:3:94:0 -> ../../sdg pci-0000:86:00.0-scsi-0:3:94:0 -> ../../sdh | pci-0000:86:00.0-scsi-0:3:95:0 -> ../../sdh I'm still not sure what causes the target_numbers to be different. However, the existence of such corner cases makes /dev/disk/by-path unusable for the volume provisioner, similar to /dev/disk/by-id/. So, If it's not possible to configure disk serialization detection, then it seems that implementing predictable disk names is the only option. Jianlin > > Thanks, > > Bart.
