Hi Richard,
On Tue, 2025-02-25 at 16:57 +0100, Richard Weinberger wrote:
> Hello!
>
> As I understand it, the current Device Mapper concept requires that
> udev
> be up and running while a new DM device is being configured, as
> mandated by
> all udev rules.
>
> I'd like to highlight a case where this requirement is not met and
> explore
> possible workarounds or solutions.
>
> Configuring DM devices, such as dm-crypt, before systemd (and udev)
> starts
> is common in embedded systems. This typically occurs within a
> minimalist
> initramfs or via the dm-mod.create= kernel parameter.
>
> In most cases, this issue goes unnoticed because users are only
> configuring
> the root filesystem. However, our scenario is different: all dm-crypt
> devices are configured before real userspace starts, and userspace
> later
> mounts them in various ways. These devices are also listed in
> /etc/fstab,
> which is processed by systemd.
>
> Since the DM device is created before udev is up, the cold-plugged DM
> has the following
> properties:
>
> $ udevadm info /dev/mapper/cr_sdb
> P: /devices/virtual/block/dm-0
> M: dm-0
> R: 0
> U: block
> T: disk
> D: b 252:0
> N: dm-0
> L: 0
> Q: 12
> E: DEVPATH=/devices/virtual/block/dm-0
> E: DEVNAME=/dev/dm-0
> E: DEVTYPE=disk
> E: DISKSEQ=12
> E: MAJOR=252
> E: MINOR=0
> E: SUBSYSTEM=block
> E: USEC_INITIALIZED=56919818
> E: DM_UDEV_DISABLE_SUBSYSTEM_RULES_FLAG=1
> E: DM_UDEV_DISABLE_DISK_RULES_FLAG=1
> E: DM_UDEV_DISABLE_OTHER_RULES_FLAG=1
> E: SYSTEMD_READY=0
> E: TAGS=:systemd:
> E: CURRENT_TAGS=:systemd:
>
> As a result, only the raw DM device is present, but systemd does not
> recognize
> it as ready. Consequently, the mount unit dependency is missing,
> causing the
> boot process to fail.
>
> In a discussion with Martin Wilck, he suggested that suspending and
> resuming
> the DM device might trigger the appropriate udev rule processing.
> Thanks a lot
> for that!
>
> Indeed, after running:
>
> dmsetup suspend /dev/mapper/cr_sdb && dmsetup resume
> /dev/mapper/cr_sdb
>
> udev properly processes the DM device, and systemd recognizes it as
> ready:
>
> $ udevadm info /dev/mapper/cr_sdb
> P: /devices/virtual/block/dm-0
> M: dm-0
> R: 0
> U: block
> T: disk
> D: b 252:0
> N: dm-0
> L: 0
> S: mapper/cr_sdb
> S: disk/by-id/dm-name-cr_sdb
> S: disk/by-uuid/f1b7c2ce-55cd-4f85-a192-cb43f55a781d
> S: disk/by-id/dm-uuid-CRYPT-PLAIN-
> cr_sdb
>
>
> Q:
> 12
>
>
> E: DEVPATH=/devices/virtual/block/dm-
> 0
>
>
> E: DEVNAME=/dev/dm-
> 0
>
>
> E:
> DEVTYPE=disk
>
>
> E:
> DISKSEQ=12
>
>
> E:
> MAJOR=252
>
>
> E: MINOR=0
> E: SUBSYSTEM=block
> E: USEC_INITIALIZED=56919818
> E: DM_UDEV_DISABLE_LIBRARY_FALLBACK_FLAG=1
> E: DM_UDEV_PRIMARY_SOURCE_FLAG=1
> E: DM_ACTIVATION=1
> E: DM_NAME=cr_sdb
> E: DM_UUID=CRYPT-PLAIN-cr_sdb
> E: DM_SUSPENDED=0
> E: DM_UDEV_RULES_VSN=2
> E: ID_FS_UUID=f1b7c2ce-55cd-4f85-a192-cb43f55a781d
> E: ID_FS_UUID_ENC=f1b7c2ce-55cd-4f85-a192-cb43f55a781d
> E: ID_FS_VERSION=1.0
> E: ID_FS_BLOCKSIZE=4096
> E: ID_FS_LASTBLOCK=131072
> E: ID_FS_SIZE=536870912
> E: ID_FS_TYPE=ext4
> E: ID_FS_USAGE=filesystem
> E: DEVLINKS=/dev/mapper/cr_sdb /dev/disk/by-id/dm-name-cr_sdb
> /dev/disk/by-uuid/f1b7c2ce-55cd-4f85-a192-cb43f55a781d /dev/disk/by-
> id/dm-uuid-CRYPT-PLAIN-cr_sdb
> E: TAGS=:systemd:
> E: CURRENT_TAGS=:systemd:
As discussed on IRC, the point is "DM_UDEV_PRIMARY_SOURCE_FLAG=1".
This flag indicates to the dm udev rules that the device had previously
been set up correctly. The flag is only set for uevents that originate
from a genuine libdm operation. Under normal circumstances (when udev
is running in the initrd), the flag is saved in the udev db, and
because the rules set the "db_persist" flag for DM devices, it will
still be present after switching to the real root. Therefore, even
though the "coldplug" uevents after switching root are "synthetic"
events generated by "udevadm trigger", they will carry this flag, and
the rules will activate the device.
In your case, there is no existing udev db entry for the device after
switching root because udev wasn't running in the initrd, and thus the
IMPORT{db} commands in 10-dm.rules have no effect. The coldplug events
will be discarded by the rules as pointless synthetic events.
The suspend/resume trick helps because the flag will be set and the
rules will then activate the device as desired.
> While I can live with this workaround, I'd like to ask for further
> input.
> Is there a simpler solution available?
You could fake a simple udev db entry in the initrd after setting up
the device.
Or you rewrite "10-dm.rules", in particular the logic that determines
whether a device has been set up correctly, to suit your needs. You
should find some way to communicate to the rules that the device has
been properly set up in the initrd (thinking about it, that's not much
different from faking an udev db entry).
I wouldn't recommend disabling the logic for discerning "synthetic" add
events completely.
> I'm also open to implementing changes in either the kernel or udev
> rules
> to better handle this corner case.
Feel free to come up with patches. But please note that the current
logic has been carefully designed and evolved over decades. DM devices
are different from almost all other devices because the "add" event
for these devices is usually meaningless, as only an empty container
with no targets is added. That's the main reason why dm devices need
special treatment in udev. It's true that these rules have been crafted
for setups where udev is already running in the initrd. It might be
worthwhile to come up with an alternative rule set for setups where
this isn't the case.
(IIRC in ancient past we had initrds without udev in mainstream
distributions, perhaps you can find working code there).
> If no alternative exists, I'd kindly ask that this workaround be
> recognized
> and preserved to ensure that unconventional DM setups, such as those
> in
> embedded systems, continue to function properly.
How do you propose to "recognize and preserve" it? Should comments
be added to the rules files or to the documentation?
Regards
Martin
