Re: tee-supplicant initrd startup before tpm2.target and dev-tpmrm0.device

[Mikko Rapeli <mikko.rapeli@xxxxxxxxxx>]

Hi,

On Thu, May 23, 2024 at 06:00:07PM +0200, Lennart Poettering wrote:
> On Do, 23.05.24 10:54, Mikko Rapeli (mikko.rapeli@xxxxxxxxxx) wrote:
> 
> > Hi,
> >
> > I'm running in circles and failing to start optee userspace daemon tee-supplicant
> > correctly with systemd in initrd.
> >
> > In certain firmware/HW configurations with optee and firmware TPM trusted application,
> > the setup needs tee-supplicant to start in initrd userspace before the fTPM kernel
> > module gets enumerated, but I'm failing to express this in the systemd
> > service dependencies.
> >
> > TPM usage in firmware is being detected correctly and tpm2.target is queued correctly,
> > but the dev-tpmrm0.device is not found since tee-supplicant@teepriv0.service is not
> > getting started before it.
> >
> > optee kernel driver is loaded and working. /dev/teepriv0 is
> > generated by udev but not
> 
> Note that udev does not generate device nodes. The kernel does. udev
> just chmods/chown/acls it and maintains metadata about it.
> 
> > before dev-tpmrm0.device.
> >
> > tee-supplicant@.service:
> >
> > [Unit]
> > Description=TEE Supplicant on %i
> >
> > [Service]
> > User=root
> 
> This line is redundant.

Will remove, thanks.

> > EnvironmentFile=-@sysconfdir@/default/tee-supplicant
> > ExecStart=@sbindir@/tee-supplicant $OPTARGS
> >
> > [Install]
> > WantedBy=basic.target
> 
> Usually you'd hook services into "sysinit.target" not
> "basic.target". The job of "basic.target" is really do combine
> sysinit.target (i.e. early-boot services), local-fs.target
> (i.e. local mounts), swaps.target (swaps), sockets.target (well, you
> guess it), and so on.
> 
> Hence, if you plug in services, use sysinit.target.

tee-supplicant@.service needs to run early, before rootfs decryption in case of fTPM.
Would "WantedBy=sysinit.target" and "Before: tpm2.target" be correct then?

> > udev rule is:
> >
> > KERNEL=="tee[0-9]*", MODE="0660", OWNER="root", GROUP="teeclnt", TAG+="systemd"
> >
> > # If a /dev/teepriv[0-9]* device is detected, start an instance of
> > # tee-supplicant.service with the device name as parameter
> > KERNEL=="teepriv[0-9]*", MODE="0660", OWNER="root", GROUP="teeclnt", \
> >     TAG+="systemd", ENV{SYSTEMD_WANTS}+="tee-supplicant@%k.service"
> >
> > So basically dev-tpmrm0.device depends on tee-supplicant@teepriv0.service started
> > on dev-teepriv0.device by udev. How to express this dependency?
> 
> I am not sure I grok this dependency chain?
> 
> What do you mean by ordering the service against dev-tpmrm0.device?
> why would you order this? I mean, when
> tee-supplicant@teepriv0.service is invoked it will do its thing and
> synthesize a /dev/tpmrm0, right?

Firmware has optee and an fTPM trusted app. In kernel we have the fTPM driver
but all of these depend also on the userspace tee-supplicant to handle optee IPC setup
and possibly RPMB emulation (in my case I have kernel patches for this, and native
support on the platforms). So the fTPM TA in optee only enumerates after tee-supplicant
has been started, and this in turn triggers fTPM kernel module loading with udev.

> Generally, you cannot really order device units, it's not under
> systemd's unit engine's control when they show up. They show up when
> user plugs in a device, or udev triggers a device or the kernel
> otherwise probes and makes a device available, and that can be any
> time. So we can *wait* for devices, and we can sometimes call tools
> that synthesize synthetic devices, but we cannot order arbitrary
> devices, that simply is out of our control.

tpm2.target works and it waits for the /dev/tpmrm0 device correctly. I need to
queue tee-supplicant startup to happen before or during that time.

> > I tried to queue tee-supplicant@.service with "Wants: tpm2.target" but that did not work
> > and seems wrong. The dependency is earlier to the kernel /dev/tpmrm0 device node.
> > Then I tried to amend the teepriv udev rule to
> > ENV{SYSTEMD_WANTS}+="tee-supplicant@%k.service tpm2.target" and
> > ENV{SYSTEMD_BEFORE}+="tpm2.target" but this did not work either. I must be doing this
> > somehow wrong. Any ideas what would work?
> >
> > Example serial log from a rockpi4b board where fTPM is failing to be detected in
> > initramfs since tee-supplicant wasn't started:
> > https://ledge.validation.linaro.org/scheduler/job/87532
> 
> This shows an ordering cycle. Address that first. If you have an
> ordering cycle systemd will drop jobs from the initial transactions in
> an attempt to fix it, but it's not always clear that the one it drops
> it the best one to drop.

Indeed, these ordering problems may be the root cause.

> The logs do not show that your "tee-supplicant@.service" unit gets
> enqueued. So are you sure your udev rule even works?

The udev rule works. With debug logs I see:

https://ledge.validation.linaro.org/scheduler/job/87556

[0;38;5;245mtee-supplicant@teepriv0.service: starting held back, waiting for: basic.target[0m

So I think I need to disable the default dependencies with DefaultDependencies=no
With my limited understanding I think I would also need the tpm2.target to wait for
tee-supplicant startup. "WantedBy=sysinit.target tpm2.target" would do it?
Or alternatively in tpm2.target

After=dev-tpmrm0.device dev-teepriv0.device
Wants=dev-tpmrm0.device dev-teepriv0.device

But the latter is wrong on devices without optee but with tpm2 device.
I must be getting some of these somehow backwards.

tpm2.target really just needs to wait for dev-tpmrm0.device but in case the firmware has optee,
the tee-supplicant@teepriv0.service should be started before tpm2.target since it may be needed
for the TPM device to show up.

Cheers,

-Mikko