On Tue, 2022-09-13 at 17:37 +0300, Konstantin Kharlamov wrote: > Hello! Sorry if it's the wrong list, it is referred to by multipath-tools > without saying it's "devel", so I presume user questions are okay…? > > How to debug a lack of devices in `multipath -l` and `… -ll` outputs? I'm > studying `multipathd -dv5` logs, but it's not at all clear to me as to what to > look for. > > At $WORK I'm trying to replace the older Ubuntu shipped multipath-tools with > the > upstream one. Then, given a system with the upstream `multipath-tools` > preinstalled and `multipathd` running, none of the `multipath -l` or > `multipath > -ll` show output. Then, if I downgrade `multipath-tools` to the one from > Ubuntu > repository and restart `multipathd`, devices do appear. > > Then it gets weird: if I upgrade `multipath-tools` back to the latest one > upstream, and reboot, devices are still there! It starts working! > > At this point I found the `/etc/multipath/wwids` which might be the reason. I > presume `multipathd` puts devices there under some conditions. But where can I > see how it decides whether to work with devices or to ignore them? I'm looking > through -v5 startup logs when it's working and when it doesn't, there's a lot > of > output, but it's hard to `diff` the older (working) and the newer (not > working) > output — a lot of style changes. Entries I find suspicious are repeated in > both > cases. > > I figured, it might ask here for pointers, is there something specific to look > for? I found the problem, and in retrospective I doubt the logs would have helped me. The problem was that we relied on default value of `find_multipaths`, but then upstream changed it, which made everything silently break on update. May I grunt: that upstream change was a terrible decision, because if you have a system with "wwids" file, it will actually continue working… For some time. Until problems with missing devices would silently creep up on you. We got very lucky here, since our infrastrcture allowed to spot that immediately. Anyway, what's done is done, and the change is dated by 2018 year, so oh well… C'est la vie. So, anyway, returning to my original question: I'm not sure I have had any chance to figure that problem out from the logs. I mean, the logs do mention something like "device foo was skipped as it's not in wwids file", but at the same time I know that the older version recreates the wwids file. And actually, even the `wwids` file say at the top a "This file is automatically generated and thou shalt not touch it". So there's no reason to assume wwids file has anything to do with the problem whatsoever. In my case what helped was the good ol' friend git-bisect. -- dm-devel mailing list dm-devel@xxxxxxxxxx https://listman.redhat.com/mailman/listinfo/dm-devel
