On Sat, May 14, 2022 at 6:55 AM Lukas Wunner <lukas@xxxxxxxxx> wrote: > > On Wed, May 11, 2022 at 12:43:34PM -0700, Dan Williams wrote: > > On Wed, May 11, 2022 at 12:20 PM Lukas Wunner <lukas@xxxxxxxxx> wrote: > > > But the reset argument still stands: That same section says that all > > > IDE streams transition to Insecure and all keys are invalidated upon > > > reset. > > > > Right, this isn't the only problem with reset vs ongoing CXL operations... > > > > https://lore.kernel.org/linux-cxl/164740402242.3912056.8303625392871313860.stgit@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/ > > The above-linked cover letter refers to AER. > > I believe with AER, the kernel is notified of an error via an interrupt > and asynchronously attempts recovery through a reset. > Obviously, an eternity may pass until the kernel gets around to do that > and whether accesses performed between the initial error and the reset > succeed is sort of undefined. So it's kind of a "best effort" error > recovery. > > With the advent of DPC, the situation has improved considerably as the > hardware (not the kernel) automatically disables the link upon occurrence > of the initial error. DPC, as far is I can see, is broken for CXL, any link going down causes the entire active interleaved memory range to be lost. Hence the "hopeful" designation in that patch set, if the link is going down to DPC the chance that the kernel runs long enough to even report the error is at risk. > Any subsequent accesses will fail and the kernel > does not perform a reset itself (the hardware already did that) but merely > attempts to bring the link back up. That has made error recovery pretty > solid and NVMe drives now seamlessly recover from errors without the need > to unbind/rebind the driver. Data centers heavily depend on that feature. Works great for NVME. > > Perhaps if CXL.mem used DPC, it would be able to recover more reliably? So far all I can see are attempts to fail a bit more gracefully, but I would not consider this a reliable architecture for recovery: https://www.computeexpresslink.org/_files/ugd/0c1418_f63f7f1a9f474ba2b00f5e77429867cb.pdf > Circling back to the SPDM/IDE topic, while NVMe is now capable of > reliably recovering from errors, it does expect the kernel to handle > recovery within a few seconds. I'm not sure we can continue to > guarantee that if the kernel depends on user space to perform > re-authentication with SPDM after reset. That's another headache > that we could avoid with in-kernel SPDM authentication. What is missing from this conversation is what constitutes a device leaving the trusted compute boundary and is the existing attestation invalidated by a reset. I.e. perhaps the kernel can just do a keep-alive heartbeat after the reset with the already negotiated key to confirm the session is still valid.
