On 28/2/25 20:48, Aneesh Kumar K.V wrote:
Xu Yilun <yilun.xu@xxxxxxxxxxxxxxx> writes:
On Thu, Feb 27, 2025 at 07:27:22PM +0530, Aneesh Kumar K.V wrote:
Xu Yilun <yilun.xu@xxxxxxxxxxxxxxx> writes:
On Wed, Feb 26, 2025 at 05:40:02PM +0530, Aneesh Kumar K.V wrote:
Xu Yilun <yilun.xu@xxxxxxxxxxxxxxx> writes:
On Fri, Feb 21, 2025 at 01:43:28PM +0530, Aneesh Kumar K.V wrote:
Alexey Kardashevskiy <aik@xxxxxxx> writes:
....
I am trying to wrap my head around your tsm. here is what I got in my tree:
https://github.com/aik/linux/blob/tsm/include/linux/tsm.h
Shortly:
drivers/virt/coco/tsm.ko does sysfs (including "connect" and "bind" to
control and "certs"/"report" to attest) and implements tsm_dev/tsm_tdi,
it does not know pci_dev;
drivers/pci/tsm-pci.ko creates/destroys tsm_dev/tsm_dev using tsm.ko;
drivers/crypto/ccp/ccp.ko (the PSP guy) registers:
- tsm_subsys in tsm.ko (which does "connect" and "bind" and
- tsm_bus_subsys in tsm-pci.ko (which does "spdm_forward")
ccp.ko knows about pci_dev and whatever else comes in the future, and
ccp.ko's "connect" implementation calls the IDE library (I am adopting
yours now, with some tweaks).
tsm-dev and tsm-tdi embed struct dev each and are added as children to
PCI devices: no hide/show attrs, no additional TSM pointer in struct
device or pci_dev, looks like:
aik@sc ~> ls /sys/bus/pci/devices/0000:e1:04.0/tsm-tdi/tdi:0000:e1:04.0/
device power subsystem tsm_report tsm_report_user tsm_tdi_bind
tsm_tdi_status tsm_tdi_status_user uevent
aik@sc ~> ls /sys/bus/pci/devices/0000:e1:04.0/tsm_dev/
device power subsystem tsm_certs tsm_cert_slot tsm_certs_user
tsm_dev_connect tsm_dev_status tsm_meas tsm_meas_user uevent
aik@sc ~> ls /sys/class/tsm/tsm0/
device power stream0:0000:e1:00.0 subsystem uevent
aik@sc ~> ls /sys/class/tsm-dev/
tdev:0000:c0:01.1 tdev:0000:e0:01.1 tdev:0000:e1:00.0
aik@sc ~> ls /sys/class/tsm-tdi/
tdi:0000:c0:01.1 tdi:0000:e0:01.1 tdi:0000:e1:00.0 tdi:0000:e1:04.0
tdi:0000:e1:04.1 tdi:0000:e1:04.2 tdi:0000:e1:04.3
SPDM forwarding seems a bus-agnostic concept, "connect" is a PCI thing
but pci_dev is only needed for DOE/IDE.
Or is separating struct pci_dev from struct device not worth it and most
of it should go to tsm-pci.ko? Then what is left for tsm.ko? Thanks,
For the Arm CCA DA, I have structured the flow as follows. I am
currently refining my changes to prepare them for posting. I am using
tsm-core in both the host and guest. There is no bind interface at the
sysfs level; instead, it is managed via the KVM ioctl
Host:
step 1.
echo ${DEVICE} > /sys/bus/pci/devices/${DEVICE}/driver/unbind
echo vfio-pci > /sys/bus/pci/devices/${DEVICE}/driver_override
echo ${DEVICE} > /sys/bus/pci/drivers_probe
step 2.
echo 1 > /sys/bus/pci/devices/$DEVICE/tsm/connect
step 3.
using VMM to make the new KVM_SET_DEVICE_ATTR ioctl
+ dev_num = vfio_devices[i].dev_hdr.dev_num;
+ /* kvmtool only do 0 domain, 0 bus and 0 function devices. */
+ guest_bdf = (0ULL << 32) | (0 << 16) | dev_num << 11 | (0 << 8);
+
+ struct kvm_vfio_tsm_bind param = {
+ .guest_rid = guest_bdf,
+ .devfd = vfio_devices[i].fd,
+ };
+ struct kvm_device_attr attr = {
+ .group = KVM_DEV_VFIO_DEVICE,
+ .attr = KVM_DEV_VFIO_DEVICE_TDI_BIND,
+ .addr = (__u64)¶m,
+ };
+
+ if (ioctl(kvm_vfio_device, KVM_SET_DEVICE_ATTR, &attr)) {
+ pr_err("Failed KVM_SET_DEVICE_ATTR for KVM_DEV_VFIO_DEVICE");
+ return -ENODEV;
+ }
+
I think bind (which brings device to a LOCKED state, no MMIO, no DMA)
cannot be a driver agnostic behavior. So I think it should be a VFIO
ioctl.
For the current CCA implementation bind is equivalent to VDEV_CREATE
which doesn't mark the device LOCKED. Marking the device LOCKED is
driven by the guest as shown in the steps below.
Could you elaborate why vdev create & LOCK can't be done at the same
time, when guest requests "lock"? Intel TDX also requires firmware calls
like tdi_create(alloc metadata) & tdi_bind(do LOCK), but I don't see
there is need to break them out in different phases.
Yes, that is possible and might be what I will end up doing. Right now
I have kept the interface flexible enough as I am writing these changes.
Good to know that, thanks.
Device can possibly be presented in locked state to the guest.
This is also what I did before. But finally I dropped (or pending) this
"early binding" support. There are several reset operations during VM
setup and booting, especially the ones in bios. They breaks LOCK state
and I have to make VFIO suppress the reset, or reset & recover, but that
seems not worth the effort.
May wanna know how you see this problem.
Currently, my approach involves a split vdev_create and a TDISP lock, which is
why I haven't encountered the issue mentioned above. The current changes
implement vdev_create via the VMM, while the guest makes an RSI call to
switch the device to the locked state.
I chose to separate vdev_create and TDISP lock into two distinct steps
to simplify the process and better align it with the RMM spec [1].
I noticed that SEV-TIO performs a KVM_EXIT_VMGEXIT, which carries out a
similar operation unless it has already been handled during VM startup.
From your reply above, I understand there was a proposal to combine
VDEV_CREATE and TDISP_LOCK. However, you also mentioned that if we
present the device in a locked state to a VM early in the boot process,
we might unintentionally break the TDISP lock state.
Linux will break it (or/and my device are buggy?), I have this in my stash:
https://github.com/aik/linux/commit/805d6763d349be173a93a4411912c4763ab44c60
"RFC: PCI: Avoid needless touching of Command register"
I will look up the previous discussions to better understand the
rationale behind combining vdev_create and lock.
It is the opposite, there is no obvious reason to separate these so we
are trying to make things simpler for now.
[1] https://developer.arm.com/-/cdn-downloads/permalink/Architectures/Armv9/DEN0137_1.1-alp12.zip
ah this is the spec. "Realm Management Monitor" is not very obvious
name when searching for TDISP (but I did not try hard :) ) Thanks,
-aneesh
--
Alexey
