how to use systemd-sysext addons and systemd-stub to extend an UKI initrd

[Emanuele Giuseppe Esposito <eesposit@xxxxxxxxxx>]

Hello everyone,

As the title suggests, I am trying to extend an UKI initrd via
systemd-sysext addons/extensions.

I contributed to the systemd-stub UKI addons to extend the kernel
command line, so I know how they works and planning to give a talk about
them soon. However, I would like to get the full picture by using the
same mechanism but with systemd-sysext addons to extend also initrd.

As I understood, a systemd-sysext addon in
/boot/efi/EFI/Linux/<UKI_NAME>.efi.extra.d will be put in /.extra/sysext
by systemd-stub, and then will be picked up by systemd-sysext to be
added into the initrd.

I am using Fedora, I created my UKI devel.efi, and made sure (just for
safety) that the initrd contains the systemd-sysext module, as I
generated it with dracut.

The UKI is created with freshly compiled systemd-stub from commit
5808300c44. Kernel is 6.6.0-0.rc1.20230915git9fdfb15a3dbf.17.fc40.x86_64

Then, I created a super dumb extension and put it in the right location:
    mkdir extension
    cd extension/
    vi ciao.txt
    mkdir usr
    cp ciao.txt usr/ciao2.txt
    cat /etc/os-release
    mkdir -p usr/lib/extension-release.d/
    echo ID=fedora > usr/lib/extension-release.d/extension-release.extension
    echo VERSION_ID=40 >>
usr/lib/extension-release.d/extension-release.extension
    cat usr/lib/extension-release.d/extension-release.extension
    cd ..
    mksquashfs extension extension.raw
    mv extension.raw /boot/efi/EFI/Linux/devel.efi.extra.d/

At this point, I set the UKI to be used at next boot and rebooted.
However, as you can imagine, this is not enough to make it work. Upon
inspecting systemd-sysext status, I see

# systemctl status systemd-sysext --no-pager -l
× systemd-sysext.service - Merge System Extension Images into /usr/ and
/opt/
     Loaded: loaded (/usr/lib/systemd/system/systemd-sysext.service;
disabled; preset: disabled)
    Drop-In: /usr/lib/systemd/system/service.d
             └─10-timeout-abort.conf
     Active: failed (Result: exit-code) since Mon 2023-12-04 11:02:15
EST; 30min ago
  Condition: start condition unmet at Mon 2023-12-04 11:16:55 EST; 15min ago
             ├─ ConditionDirectoryNotEmpty=|/etc/extensions was not met
             ├─ ConditionDirectoryNotEmpty=|/run/extensions was not met
             ├─ ConditionDirectoryNotEmpty=|/var/lib/extensions was not met
             └─ ConditionDirectoryNotEmpty=|/.extra/sysext was not met
       Docs: man:systemd-sysext.service(8)
   Main PID: 479 (code=exited, status=1/FAILURE)
        CPU: 11ms

Dec 04 11:02:15 localhost systemd-sysext[479]: Failed to read metadata
for image extension: No such device
Dec 04 11:02:15 localhost systemd[1]: systemd-sysext.service: Main
process exited, code=exited, status=1/FAILURE
Dec 04 11:02:15 localhost systemd[1]: systemd-sysext.service: Failed
with result 'exit-code'.
Dec 04 11:02:15 localhost systemd[1]: Failed to start
systemd-sysext.service - Merge System Extension Images into /usr/ and /opt/.
Dec 04 11:16:55 ibm-p8-kvm-03-guest-02.virt.pnr.lab.eng.rdu2.redhat.com
systemd[1]: systemd-sysext.service - Merge System Extension Images into
/usr/ and /opt/ was skipped because no trigger condition checks were met.

So question number 1: what went wrong here? How do I debug it?

---

Next question is (and probably next problem too), how to actually sign
an extension?
According to the --image-policy doc, I see

When run in the initrd and operating on a system extension image stored
in the /.extra/sysext/ directory a slightly stricter policy is used by
default: "root=signed+absent:usr=signed+absent"

Which means this .raw (assuming I built it correctly) needs to be
signed. How can I sign a disk image? I tried looking for dm-verify but I
don't understand how to do that, nor if there is a systemd tool that can
do it.

I am hoping to find something like ukify to create and sign sysext
addons very easily.

---

Supposing I manage to do all of the above, my next question would be
how/if to override the /lib folder instead of the traditional /usr or
/opt, as for example I might want to add another kernel module into the UKI.

---

Last but not least is where is the documentation for this. I couldn't
find anything at all about systemd-sysext, and therefore I would be very
very happy to write (other than presenting it) some doc to make the life
easier to anyone like me that is looking forward to using these new
features.

Thank you in advance and sorry for the wall of text,
Emanuele