Re: A path to a different device was added to an existing mapth

Ilan Steinberg <ilan.steinberg@xxxxxxxxxxxxx> · Thu, 19 Mar 2015 10:35:26 +0000

Hi,

I have some more info:

multipathd> show maps

name    sysfs uuid

mpathhs dm-2  20024f400d5190000

mpathit dm-16 20024f400d519002b

mpathis dm-15 20024f400d519001c

mpathin dm-11 20024f400d5190012

mpathim dm-14 20024f400d5190011

mpathiw dm-13 20024f400d5190010

mpathhz dm-7  20024f400d5190006

mpathhx dm-6  20024f400d5190005

mpathhw dm-5  20024f400d5190004

mpathht dm-8  20024f400d5190003

mpathij dm-12 20024f400d519000e

mpathil dm-10 20024f400d519000d

mpathhv dm-3  20024f400d5190001

mpathik dm-9  20024f400d519000f

mpathhu dm-4  20024f400d5190002

It seems that mpathiw is indeed with the uuid of sdag but...

note that  if you run 'multipath -l' you will see mpathiw having the correct

signature (20024f400d5190026) - because it queries it again from the storage.

Also:

multipathd> show paths format "%d %s %S %w"

dev  vend/prod/rev size uuid

.

.

sdaf KMNRIO,K2     300G 20024f400d519001e

.

sdw  KMNRIO,K2     100G 20024f400d5190010

.

.

sdz  KMNRIO,K2     100G 20024f400d5190026

.

sdag KMNRIO,K2     300G 20024f400d5190010

Unfortunately, the device for sdag was deleted from the storage, but I can still see that:

[root@il-ksm3-044 ~]# sg_inq -p 0x83 /dev/sdw

VPD INQUIRY: Device Identification page

  Designation descriptor number 1, descriptor length: 36

    designator_type: T10 vendor identification,  code_set: ASCII

    associated with the addressed logical unit

      vendor id: KMNRIO  

      vendor specific: ATF-VG-1:EXP-SNAd5190026

  Designation descriptor number 2, descriptor length: 12

    designator_type: EUI-64 based,  code_set: Binary

    associated with the addressed logical unit

      EUI-64 based 8 byte identifier

      IEEE Company_id: 0x24f4

      Vendor Specific Extension Identifier: 0xd5190026

      [0x0024f400d5190026]

[root@il-ksm3-044 ~]# sg_inq -p 0x83 /dev/sdz

VPD INQUIRY: Device Identification page

  Designation descriptor number 1, descriptor length: 36

    designator_type: T10 vendor identification,  code_set: ASCII

    associated with the addressed logical unit

      vendor id: KMNRIO  

      vendor specific: ATF-VG-1:EXP-SNAd5190026

  Designation descriptor number 2, descriptor length: 12

    designator_type: EUI-64 based,  code_set: Binary

    associated with the addressed logical unit

      EUI-64 based 8 byte identifier

      IEEE Company_id: 0x24f4

      Vendor Specific Extension Identifier: 0xd5190026

      [0x0024f400d5190026]

So, sdw and sdz are path to the same device, and so are sdaf and sdag...

Regards,

Ilan Steinberg

From: Ilan Steinberg

Sent: Thursday, March 19, 2015 12:10 PM

To: dm-devel@xxxxxxxxxx

Cc: Eli Malul; Guy Keren; Shahar Salzman

Subject: A path to a different device was added to an existing mapth

Hello,

I have never posted here before but I can across an interesting error.

I have run into a situation were a path (sdag) to LUN 15 with a scsi serial number of ending with 0xd5190010 was added to an mapth (mpathiw) which is for LUN 11 with scsi serial number ending with 0xd5190026 (sdz
 and sdw)

I would appreciate any help understanding how/why this happened.

Initiator Info:

Linux il-ksm3-044 2.6.32-431.el6.x86_64 #1
device-mapper-multipath version 0.4.9

multipath.conf:
defaults {

                   user_friendly_names     yes

                   polling_interval        10

                    verbosity               2

}

devices {

                    device  {

                             vendor                      "KMNRIO"

                             product                     "K2"

                         path_grouping_policy        multibus

                             getuid_callout          "/lib/udev/scsi_id --whitelisted --device=/dev/%n"

                             path_checker                tur

                             path_selector               "queue-length 0"

                             no_path_retry               fail

                             hardware_handler            "0"

                             rr_weight                   priorities

                             rr_min_io               1

                             failback                15

                             fast_io_fail_tmo        5

                             dev_loss_tmo            8

                             }

}

Since I see that sdz and sdw used to belong to mapthiv (was it renamed to mathiw?)

This might be useful info - in the bindings file I see:
...
mpathiv 20024f400d5190010                      <---------------------------------------- this is the scsi_sn of sdag
mpathiw 20024f400d5190026                     <---------------------------------------- this is the scsi_sn of sdz/w
...

I am adding the relevant host logs (I can add the full logs if needed) - error is in the last log line:

Mar 14 09:44:36 il-ksm3-044 kernel: sdag: detected capacity change from 0 to 322122547200

Mar 14 09:59:35 il-ksm3-044 kernel: sd 8:0:1:15: [sdag] Warning! Received an indication that the LUN reached a thin provisioning soft threshold.

Mar 14 10:00:06 il-ksm3-044 kernel: sd 8:0:1:15: [sdag] Warning! Received an indication that the LUN reached a thin provisioning soft threshold.

Mar 14 10:00:21 il-ksm3-044 kernel: sd 8:0:1:15: [sdag] Warning! Received an indication that the LUN reached a thin provisioning soft threshold.

Mar 14 10:26:31 il-ksm3-044 kernel: end_request: I/O error, dev sdz, sector 167369280
Mar 14 10:26:31 il-ksm3-044 kernel: end_request: I/O error, dev sdz, sector 80032448

Mar 14 10:26:44 il-ksm3-044 multipathd: sdz: remove path (uevent)
Mar 14 10:26:44 il-ksm3-044 multipathd: sdz [65:144]: path removed from map mpathiv
Mar 14 10:26:44 il-ksm3-044 multipathd: sdag: remove path (uevent)
Mar 14 10:26:44 il-ksm3-044 multipathd: sdag [66:0]: path removed from map mpathks

Mar 14 10:26:44 il-ksm3-044 multipathd: mpathiv: load table [0 209715200 multipath 0 0 1 1 queue-length 0 1 1 65:96 1]
Mar 14 10:26:44 il-ksm3-044 multipathd: mpathks: load table [0 629145600 multipath 0 0 1 1 queue-length 0 1 1 65:240 1]

Mar 14 10:53:50 il-ksm3-044 kernel: sd 8:0:1:11: [sdz] 209715200 512-byte logical blocks: (107 GB/100 GiB)
Mar 14 10:53:50 il-ksm3-044 kernel: sd 8:0:1:11: [sdz] Write Protect is off
Mar 14 10:53:50 il-ksm3-044 kernel: sd 8:0:1:11: [sdz] Write cache: disabled, read cache: enabled, supports DPO and FUA
Mar 14 10:53:50 il-ksm3-044 kernel: sdz:
Mar 14 10:53:50 il-ksm3-044 kernel: sd 8:0:1:11: [sdz] Attached SCSI disk
Mar 14 10:53:50 il-ksm3-044 kernel: sd 8:0:1:15: [sdag] 629145600 512-byte logical blocks: (322 GB/300 GiB)
Mar 14 10:53:50 il-ksm3-044 kernel: sd 8:0:1:15: [sdag] Write Protect is off
Mar 14 10:53:50 il-ksm3-044 kernel: sd 8:0:1:15: [sdag] Write cache: disabled, read cache: enabled, supports DPO and FUA
Mar 14 10:53:50 il-ksm3-044 kernel: sdag: unknown partition table
Mar 14 10:53:50 il-ksm3-044 kernel: sd 8:0:1:15: [sdag] Attached SCSI disk
Mar 14 10:53:50 il-ksm3-044 multipathd: sdag: add path (uevent)
Mar 14 10:53:50 il-ksm3-044 multipathd: mpathks: failed in domap for addition of new path sdag
Mar 14 10:53:50 il-ksm3-044 multipathd: sdz: add path (uevent)
Mar 14 10:53:50 il-ksm3-044 multipathd: sdz [65:144]: path added to devmap mpathiv

Mar 14 10:53:50 il-ksm3-044 multipathd: mpathiv: load table [0 209715200 multipath 0 0 1 1 queue-length 0 2 1 65:96 1 65:144 1]

Mar 14 12:29:12 il-ksm3-044 multipathd: sdw: remove path (uevent)
Mar 14 12:29:12 il-ksm3-044 multipathd: mpathiv: load table [0 209715200 multipath 0 0 1 1 queue-length 0 1 1 65:144 1]
Mar 14 12:29:12 il-ksm3-044 multipathd: sdw [65:96]: path removed from map mpathiv

Mar 15 12:45:59 il-ksm3-044 kernel: sd 8:0:1:11: [sdz] Warning! Received an indication that the LUN assignments on this target have changed. The Linux SCSI layer does not automatically remap LUN assignments.

Mar 14 12:59:18 il-ksm3-044 kernel: sd 7:0:1:11: [sdw] 209715200 512-byte logical blocks: (107 GB/100 GiB)
Mar 14 12:59:18 il-ksm3-044 kernel: sd 7:0:1:11: [sdw] Write Protect is off
Mar 14 12:59:18 il-ksm3-044 kernel: sd 7:0:1:11: [sdw] Write cache: disabled, read cache: enabled, supports DPO and FUA
Mar 14 12:59:18 il-ksm3-044 kernel: sd 7:0:1:11: [sdw] Attached SCSI disk

Mar 14 12:59:18 il-ksm3-044 multipathd: sdw: add path (uevent)
Mar 14 12:59:18 il-ksm3-044 multipathd: mpathiv: load table [0 209715200 multipath 0 0 1 1 queue-length 0 2 1 65:144 1 65:96 1]
Mar 14 12:59:18 il-ksm3-044 multipathd: sdw [65:96]: path added to devmap mpathiv

Mar 15 11:45:27 il-ksm3-044 kernel: sdz: detected capacity change from 0 to 107374182400

Mar 15 11:45:33 il-ksm3-044 multipathd: mpathiw: event checker started
Mar 15 11:45:33 il-ksm3-044 multipathd: mpathiw: devmap dm-13 added

Mar 15 12:28:12 il-ksm3-044 multipathd: mpathiw: event checker exit

Mar 15 12:28:15 il-ksm3-044 multipathd: mpathiw: event checker started
Mar 15 12:28:15 il-ksm3-044 multipathd: mpathiw: devmap dm-13 added

Mar 15 12:56:35 il-ksm3-044 kernel: end_request: I/O error, dev sdz, sector 204563968
Mar 15 12:56:35 il-ksm3-044 kernel: end_request: I/O error, dev sdz, sector 193484032
Mar 15 12:56:35 il-ksm3-044 multipathd: mpathiw: remaining active paths: 1

Mar 15 12:56:48 il-ksm3-044 multipathd: sdz: remove path (uevent)
Mar 15 12:56:48 il-ksm3-044 multipathd: mpathiw: load table [0 209715200 multipath 0 0 1 1 queue-length 0 1 1 65:96 1]
Mar 15 12:56:48 il-ksm3-044 multipathd: sdz [65:144]: path removed from map mpathiw
Mar 15 12:56:48 il-ksm3-044 multipathd: sdag: remove path (uevent)

Mar 15 13:27:00 il-ksm3-044 kernel: sd 8:0:1:11: [sdz] 209715200 512-byte logical blocks: (107 GB/100 GiB)
Mar 15 13:27:00 il-ksm3-044 kernel: sd 8:0:1:11: [sdz] Write Protect is off
Mar 15 13:27:00 il-ksm3-044 kernel: sd 8:0:1:11: [sdz] Write cache: disabled, read cache: enabled, supports DPO and FUA
Mar 15 13:27:00 il-ksm3-044 kernel: sdz: unknown partition table
Mar 15 13:27:00 il-ksm3-044 kernel: sd 8:0:1:11: [sdz] Attached SCSI disk
Mar 15 13:27:00 il-ksm3-044 multipathd: sdz: add path (uevent)
Mar 15 13:27:00 il-ksm3-044 multipathd: mpathiw: failed in domap for addition of new path sdz

Mar 15 15:20:41 il-ksm3-044 multipathd: mpathiw: devmap dm-13 added

Mar 15 17:38:36 il-ksm3-044 kernel: sd 8:0:1:11: [sdz] Warning! Received an indication that the LUN assignments on this target have changed. The Linux SCSI layer does not automatically remap LUN assignments.

Mar 15 18:44:54 il-ksm3-044 kernel: sd 8:0:1:15: [sdag] 629145600 512-byte logical blocks: (322 GB/300 GiB)
Mar 15 18:44:54 il-ksm3-044 kernel: sd 8:0:1:15: [sdag] Write Protect is off
Mar 15 18:44:54 il-ksm3-044 kernel: sd 8:0:1:15: [sdag] Write cache: disabled, read cache: enabled, supports DPO and FUA
Mar 15 18:44:54 il-ksm3-044 kernel: sdag: unknown partition table
Mar 15 18:44:54 il-ksm3-044 kernel: sd 8:0:1:15: [sdag] Attached SCSI disk

Mar 15 18:44:55 il-ksm3-044 multipathd: sdag: add path (uevent)
Mar 15 18:44:55 il-ksm3-044 multipathd: mpathiw: load table [0 209715200 multipath 0 0 1 1 queue-length 0 3 1 65:96 1 65:144 1 66:0 1]
Mar 15 18:44:55 il-ksm3-044 multipathd: sdag [66:0]: path added to devmap mpathiw

Thank you,

Ilan Steinberg

--
dm-devel mailing list
dm-devel@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/dm-devel