Re: tee-supplicant initrd startup before tpm2.target and dev-tpmrm0.device

[Mikko Rapeli <mikko.rapeli@xxxxxxxxxx>]

Hi,

On Thu, Jun 06, 2024 at 06:43:11PM +0200, Lennart Poettering wrote:
> On Do, 06.06.24 18:05, Mikko Rapeli (mikko.rapeli@xxxxxxxxxx) wrote:
> 
> > Hi,
> >
> > The initrd side startup and shutdown of tee-supplicant works now correctly
> > with:
> >
> > [Unit]
> > Description=TEE Supplicant on %i
> > DefaultDependencies=no
> > After=dev-%i.device
> > Wants=dev-%i.device
> > Conflicts=shutdown.target
> 
> > Before=systemd-pcrextend.socket systemd-pcrextend@.service
> > systemd-pcrfs-root.service systemd-pcrfs@.service
> > systemd-pcrmachine.service systemd-pcrphase-initrd.service
> > systemd-pcrphase-sysinit.service systemd-pcrphase.service
> > systemd-tpm2-setup-early.service systemd-tpm2-setup.service
> > tpm2.target sysinit.target shutdown.target
> 
> You cannot have deps on templates, only of instances of
> templates. i.e. Before=systemd-pcrextend@.service doesn't work.
> 
> It's also unnecessary since all those services have After=tpm2.target
> anyway, so if you order yours service before that, you have all that's
> needed. Hence all you need as Before= should be:
> 
> Before=tpm2.target sysinit.target shutdown.target
> 
> And you might want to add Wants=tpm2.target, so that if the tee
> supplicant is explicitly started you definitely get the milestone
> target pulled in too, even if usually it works the other way round.

Thanks, this seems to be enough.
 
> >
> > [Service]
> > Type=simple
> > ExitType=cgroup
> 
> This is very unusual for a system-level service. Are you sure this is OK?

Likely not. Reverted to default now. There was an issue in initrd that stopping
tee-supplicant before any of the TPM device using services caused the actions to fail.
Before=tpm2.target seems to fix this too.

> > EnvironmentFile=-@sysconfdir@/default/tee-supplicant
> > ExecStart=@sbindir@/tee-supplicant $OPTARGS
> 
> BTW, this pattern of havein /etc/default/ and then $OPTARG is
> something we despise, it's a compat kludge, don't do that on systemd,
> it has stronger tools for changing unit files, such as drop-ins and
> override units. Adding multiple levels of changes like this is not how
> you do things.
> 
> > [Install]
> 
> Just drop this line.

Done.

> > To stop the tee-supplicant@.service in initrd after all TPM users,
> > I used the long setup with Before=systemd-pcrphase-initrd etc. This
> > seems to work.
> 
> How is this supposed to work anyway? is the supplicant supposed to
> exit before initd transition, and be started anew after the
> transition?

Yes, and tee-supplicant must be started again before any of the TPM using services.
This now works for initrd start and also shutdown, but fails in main rootfs
where services like systemd-pcrmachine.service, systemd-tpm2-setup.service and
systemd-pcrfs-root.service fail since TPM device is not functional without
tee-supplicant in userspace. 

> > But I'm failing to setup the start in main rootfs correctly before any of the TPM2 device
> > using services. Since udev is starting the service somehow after initrd already did
> > it once, then how to redo the same steps earlier also on main rootfs?
> >
> > Log from a run where main rootfs side tee-supplicant is started
> > too late and some services already tried to use the TPM2 device
> > which resulted in failures:
> >
> > [initrd boot finishing up]
> >          Stopping [0;1;39mTEE Supplicant on teepriv0[0m...
> > [[0;32m  OK  [0m] Stopped [0;1;39mTEE Supplicant on teepriv0[0m.
> >
> > ^ initrd side tee-supplicant stopped almost last so that all users
> >   were shutdown already
> >
> > [[0;32m  OK  [0m] Finished [0;1;39mCleanup udev Database[0m.
> > [[0;32m  OK  [0m] Reached target [0;1;39mSwitch Root[0m.
> >          Starting [0;1;39mSwitch Root[0m...
> > [[0;32m  OK  [0m] Stopped [0;1;39mSwitch Root[0m.
> > [[0;32m  OK  [0m] Created slice [0;1;39mSlice /system/getty[0m.
> > [[0;32m  OK  [0m] Created slice [0;1;39mSlice /system/serial-getty[0m.
> > [[0;32m  OK  [0m] Created slice [0;1;39mUser and Session Slice[0m.
> > [[0;32m  OK  [0m] Started [0;1;39mDispatch Password Requests to Console Directory Watch[0m.
> > [[0;32m  OK  [0m] Started [0;1;39mForward Password Requests to Wall Directory Watch[0m.
> >          Expecting device [0;1;39m/dev/ttyAMA0[0m...
> >          Expecting device [0;1;39m/dev/ttyS2[0m...
> > [[0;32m  OK  [0m] Reached target [0;1;39mBlock Device Preparation for /dev/mapper/usr[0m.
> > [[0;32m  OK  [0m] Reached target [0;1;39mLocal Encrypted Volumes[0m.
> > [[0;32m  OK  [0m] Stopped target [0;1;39mSwitch Root[0m.
> > [[0;32m  OK  [0m] Stopped target [0;1;39mInitrd File Systems[0m.
> > [[0;32m  OK  [0m] Stopped target [0;1;39mInitrd Root File System[0m.
> > [[0;32m  OK  [0m] Reached target [0;1;39mLocal Integrity Protected Volumes[0m.
> > [[0;32m  OK  [0m] Reached target [0;1;39mPath Units[0m.
> > [[0;32m  OK  [0m] Reached target [0;1;39mRemote Encrypted Volumes[0m.
> > [[0;32m  OK  [0m] Reached target [0;1;39mRemote File Systems[0m.
> > [[0;32m  OK  [0m] Reached target [0;1;39mSlice Units[0m.
> > [[0;32m  OK  [0m] Reached target [0;1;39mSwaps[0m.
> > [[0;32m  OK  [0m] Reached target [0;1;39mLocal Verity Protected Volumes[0m.
> > [[0;32m  OK  [0m] Listening on [0;1;39mRPCbind Server Activation Socket[0m.
> > [[0;32m  OK  [0m] Listening on [0;1;39mRPCbind Server Activation Socket[0m.
> > [[0;32m  OK  [0m] Reached target [0;1;39mRPC Port Mapper[0m.
> > [[0;32m  OK  [0m] Listening on [0;1;39mSyslog Socket[0m.
> > [[0;32m  OK  [0m] Listening on [0;1;39minitctl Compatibility Named Pipe[0m.
> > [[0;32m  OK  [0m] Listening on [0;1;39mNetwork Service Netlink Socket[0m.
> > [[0;32m  OK  [0m] Listening on [0;1;39mTPM2 PCR Extension (Varlink)[0m.
> >
> > ^ this should only be started once tee-supplicant is running again
> > from main rootfs
> 
> This suggests tpm2.target hasn't been enqueued on the host system?
> Maybe you forgot to include the generator in the host system?

tpm2.target runs in initrd and also stops before switch to main
rootfs. In initrd side everything works correctly. The TPM encrypted
rootfs gets created and switchroot to it with a /usr dm-verity partition
works, but the TPM related service there fail. tee-supplicant does get
started in main rootfs but just too late.

Maybe the same tee-supplicant@.service unit started by udev can't do this
but two different units could: one for initrd and one for main rootfs.

This way both units get queued once the device is initially detected in
initrd.

tee-supplicant-initrd@.service would be started
in initrd and stopped before switch to main rootfs. Then in main
rootfs side, tee-supplicant@.service would be started before
any TPM and PCR using services. This could also be needed if arguments
to tee-supplicant binary would differ between initrd and main rootfs.
I'm not sure if the services should Conflicts each other, or just be
ordered with After: and Before to run at different times.

tee-supplicant-initrd@.service:

[Unit]
Description=TEE Supplicant on %i (initrd)
DefaultDependencies=no
After=dev-%i.device
Wants=dev-%i.device tpm2.target
Conflicts=shutdown.target tee-supplicant@teepriv0.service
Before=tpm2.target sysinit.target shutdown.target tee-supplicant@teepriv0.service initrd-switch-root.target

[Service]
Type=simple
EnvironmentFile=-@sysconfdir@/default/tee-supplicant
ExecStart=@sbindir@/tee-supplicant $OPTARGS

tee-supplicant@.service:

[Unit]
Description=TEE Supplicant on %i 
DefaultDependencies=no
After=dev-%i.device
Wants=dev-%i.device 
Conflicts=shutdown.target tee-supplicant-initrd@teepriv0.service
Before=systemd-pcrmachine.service systemd-tpm2-setup.service sysinit.target shutdown.target
After=tpm2.target initrd-switch-root.target tee-supplicant-initrd@teepriv0.service

[Service]
Type=simple
EnvironmentFile=-@sysconfdir@/default/tee-supplicant
ExecStart=@sbindir@/tee-supplicant $OPTARGS


> Please provide proper boot logs, with debug logging enabled.

Debug logging is available from here, sadly log is too big to view
nicely on the web page and has to be downloaded:

https://ledge.validation.linaro.org/scheduler/job/88420

Cheers,

-Mikko